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FOREWORD 



Extension workers need^facts upon wKich to base 
their decisions inctheir Cooperative Extension work. 
The' purpose of evaluation in Extension is to help 
obtain and interpret facts. The authors have writ- 
ten tnis manual to aid workers in the Cooperative 
Extension Service of the JJnited States to, be better 
able to understand z^d apply, the principles and 
methods of evaluation. , . 

Examples are drawn from and the applications 
are made to the Cooperative Extension Service, but 
principles and methods discussed have application ^ 
to extension-type education In other parts of the 
world and to other kinds of informal education. 

Extension cvaluatio\i means the use of the scien- 
tific approach in providing facts as, a basis for mak- 
ing decLsi6ns, drawing ..conclusions, or forming 
judgments about the organization and conduct of 
extension work. Evaluation in Extension is to the 
extension worker what agricultural and home eco-.^ 
nomics research is to farmers, distributors and 
homcmakcrs. 

All extension workers do some evaluation in their * 
day-to-day work. They make decisions, draw coiv 
elusions and form judgments. The aim of this 
manual us to help them improve their evaluation^ 
and thereby improve their' effectiveness in program 



planning, teachmg methods, use of local leadership 
and organization for extension work. 

'^Evaluation" is a state of mind in which an exten- 
sion 'worker wants better information about his 
work and uses the mformation to make his work 
mora effective. It is an investment which produces 
progress and growth in achieving the objectives for 
which an organization has been set up. 

The earlier editkms of the manual have been 
used for inservice training purposes at- Cooperative 
Extension workshops, conferences and sumn^er 
schools. The preliminary edition (1958) has been 
used in other countries " for similar purposes and 
has been translated into Spanish by the Institute 
Inter-Americano de Ciencias Agricolas de la OEA, 
Turrialba, Costa Rica. The manual has served as 
a reference guide to Cooperative Extension workers 
engaged in evaluation studies and to State leaders 
of evaluation in Cooperative Extension. 

In the present edition the authors have end*^avor- 
•ed to present. evaluation that will be most useful to 
teachers and students of evaluation, 

GLADYS GALLUP, Director - 

Division of Extension Research and Tiiaininc 

FEDERau EXTENSION SERVICE 
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; ^Chapter I 
EVALUATION - WHAT IT IS 



As County Agent Brown was driving homi^ ffom 
a night 'meeting, he reflected on the meeting vyith 
•a great deaLof-sati^ifaction. it was a -fine meeting. 
He wished all his meetings were as good^»Jr)his one. 

He recallpd the high levcrbf interbst w'fnch per-' 
sisled throughout." lk\snYiled co'ntentcdly as he 
remembered how'Jie had. to ''kill the meet-ing rather : 
.than let it die." After the meeting many stayed on 
to ask questions and talk among thentseives ^ih 
small; groups.^ • . ) * 

VVJiat a pleasant contrast Icrsome other meetings! ' 
" Meetings like this are what ma^e a county/iigent's* 
Ai'ork n\ean something. You don't mind the late- 
ness x)f the hour w'hen you Jcno\v your jol) was well 
Hone and that. people w^ere helped, '/ 
,* Under the glow of ^lis satisfaction, however, Mr. ' 
' Brown b\;gan "to asjc himself some questions. "Th'fey. ' 
^seeiiied enthusigist'ic enough, but 'what did they get^J 
out of the meeting? Was this an e'ducatiotial experi- 
ence for thqm or wais it purely entertainment? Did 
the. meeting bring about/any. desired- change in^ 
them? Or will it?' What did I really have in mind , 
, to accqitiplish by the /meeting? What were my 
objectives?'' ' ^ , ; 

Questions like these were tmalyti^al. Soil testing 
r had been the topic: He, hoped- they would -ge^t soiVie 
new infoVmation about soil testing and liming; -th'at 
the meeting would create .a., favorable attitude to- 
ward soil testing and limjng, and that they woaild 
test their 'soils^ and would lime when necessary. ^ 
Obviously, thgre could be no immediate evidence 
that they did test their soil\and .did.^pply. recoin- 
:mendecl amounts ;0f lime. »He would check on that ^ 
from time to time Sufing the summer. -However, ' 
'there were several indications that they intended 
to act on the information he had given them.« ^ 
' He remembered how they '*'pdrked up th'eiivears" 
when he showed'^them how easy it was td have, the 
soil tested, and where they could gei it done. . They 
were equally attentive, while he »f)oke about, the 
value of liming, the effect of liming on yields, and 
the relation of cost of liming- to increased . value of 
crops raised. JSome werg .^usy" making notes. 
Others raised qu^estions ;which -indicated they wwe 
appfying. th6 , explanation t6 their own situations. 
Several. asked him questions after the meeting. 
It seemed to Mr. Brown that these observations. 



indicated that many persons in the group. had l^arn- 
ed some of theahings he was trying to explain to 
^th' 1. TheyChad changed in that respect, ^^^xom 
these observations "he coul^ conclude thdt mfet of 
, them had a favprabie attitude, aijd"a fairly ^trong 
/ favorable-' attitude ^ so strong that'.many M^voxikl 
' probably-, do something about «oil testing arid liu> 
fiig. But I'lat, of goilrse, remained to be se^n. 
_ ^ As Mr. Brown. Was thinkirig along th6se lilies, he. 
^^was evalua'ting. He . never had any course^^r trains 
' irig in" evaluation. The,- way^he'was thinkii/g came 
' , natural- to .hirp. - \. " ; ' ' - I 

\* As fi?:j-di^ve, along his thoughts moved on to\vhy 

• '•this riieeting was so good'arid wh^-some otlilbr meet- 
. ^vings Were not so'good. What had he done/this time- 
to help bring about the changes in Hhdf.aeople? 

' More sifecjjficaily, whiit had he done this <time that 
,\fas different from what' he-; had, cfone/fan those, 
,occasions^vhiCh he- would like to forget? /K ^ 

Forl)ne thing, there was/that chart ^e ahd his 
* setretary^had Wrked^on/niost -of the /^fternqori'., 
/-He remembered how the expressions of the aud,-- 

• ience changed a^.he\vent through his explanation, 
aisled by.the chart showing yields Avith i'nd without 
.adequateuliming, relative" costs '^nd netjgain. 

' " ^He recalled, also, that this^vvas the tirst time he 
had ^bothered to rhake a dembnstration, When .he 
had finished showing how pasy it js t|i*take a', soif 

• sample, he saw many of. them t'akiiig'dovyn the 
address^Of the place jto send the samp/le. • Jle couli^ , 
teil he. had the^. c'Smplete;attenj:ion?p he was ■ 

' relating how t'lo results o^ the tests could be ap- . 

• plied and what J:he (^fect'on yiej4 \piild be. 

• There were spme- other; things which probably 
helped. For instance, he did much more vcareful ' 
planning 'and rehearsing for'''this//neeting than he » 
had done for ether meetings. He^elt more"«ure of 
hinisel'f at,- the meeting.' Mr! Brown realized ^that , 
aU thcfee* things contributed"^ to, ^^tting up a *^ood ' 

, te^arning situation for , his audieiyce. ' What he and- 
others « had done would influence the degree'; to 
►which the ptjople would Giiang(^ ; ^ ■' 

*^*So that iSvWhat my dfstii^/ agent infant wKen 
•.Jie said that a ^unty agent's i^b is to create learn- 
ing experiences by which people can change in the 
direction of educational ob^ie^tives,'' thought Mr. ' 
. Brown as he reached home djnA piit away his car.' 
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This s^ory illustrates that, whether or not a per- 
son is tiained in evaluation, he makes countless ' 
ovaluatiphs, anywciy. Whethei: -or not Mr., Brown, , 
\ras aware of it, his reflections on the trip home 
constitute a form of. evaluation. 

Evaluatio.n is a common act^yity in which we all 
engage everyday. We finish a meal and evaluate it 
by saying,-^r only thinking, how gQod. or bard it \yas. 
\Ve'may?go further thah evaluate the"^ result; we 
may conrfcfer what was especially good or especially 
poor alDOut it and-why. The meat was tender and 
juicy, -but tfee pot^itoes could have been cooked 
longer. . 

We re^d a book and react in some manner. It 
may be a very interesting one and we recommend 
it to ou friends. It may leavers so **cold" that we 
do not bother to finish it. " " 

What Is Evaluation? 

/ .The purpose of these illustrations is to point out 
that all of us' engage in evaluation in our day-to-day 
affairs. Trifling indeed^, is the action to which we do 

' not apply the three st&f)s in the evalUfition process 
in sojne degree of intensity. First we make^some 
observations or collect sonie information.- Then we 
apply some standards or criteria to our ob*>erva-' 
tions. Finally, we. form some judgment, di*aw.some 
conclu?^ or make some decision. 

Theselhree elements are involved in all evalua- 
tion. It caa be done so casually that we ^^e hardly • 
a\yare of doing it, such as looking out the window 
to decide whether or not to carry an UfSibrella. " At 

/the other extfOme is scientific- research in com-- . 

\plicated problems^'to get information which people 
Cfin use. Somewhere between will fall most 
kihds of evaluation undertaken by extension per- 
sonnel. ^ . ' • ' ' 
The better we do the evaluation job, the less " 
Jikely''we are to *'j;ump" to conclusions which are 
rjiisleading. The purpose of this nianual is tb'dis- ' 
cuss how the evaluation process caif be used to aid 
us in drawing better conclusions and in making 
better jdecisions in.our \vorl?.' The manual is not 
jdesigried to make the reader^ a specialist in ex'ten- 
sfon research. -Rather, it is' intended to familiarize 
liini with the 'processes of evaluation and to help 
him to make better evaluations in doing extension, 
work. ^ 

Degrees of Evaluation 

When we think of evaluation as a process of" 
collecting information as. a basis for making 'deci- 
sions, forming judgments and drawing coiiplusioris, 
we realize it has much in common witft scientific 
research. As with evaluation, ^we do scientific re-^ 
search to obtain usable info^rm^tidh; Even so, thfere 
is a great difference between our casual everyday 



evaluations (the umbrella ^decision) and scientific 
research. The difference, however, is a matter of 
degree rather than kind. It lies in the difference 
in the degree to v/hich scientific meth9d is neces- 
sary in the solution of problems.-' s " 

Degrees of anything are usually represented dn 
.a scale or continuum. Distance, can be from zero- 
inches to rQillion5 ot miles. There is no gap in the 
scale. Temperature is'also'^-a continuous series 
represented in degrees from-hot to., cold. Other 
examples are age; yomjg to old; .weight. Tight to 
heavy; quality, as in adjectives of conriparison, g[ood- 
better-best; strength of attitudes, strong 'to \Veak 
and favorable to unfavorable; skill, degrees of 
skill; knowledge, amounts 'from'' none to much; 
soundj loudness' oJ ^ourid; and even color, wave 
lengths of light 'are, expressed as. degrees. * 

Ttie examples illustrate^Hat fnanycharacteristics 
of people -are not on an **either-or'! basis; not neces- 
' sarily either presenj: or absent; "not^ either Black or 
white. There are shades. A characteristic m*ay;be 
partly present'and partly absent. Th'e degrees ofr^ 
characteristic blend into* each; other on -a' scale or 
' continuum with no sharp lines 'of distinction. 
'. In order to avoid ^emphasizing differences.be-- 
tween evaluation and research, it is more useful to 
emphasize similarities. Casual everyday evalua- 
tions can *e placed at one end of the s^ale, and 
"scientific research at the other end. All'degrees' 
ate found between the* two extremes. ^ ^ 

I' rusu.n Vvciryday ' , ■ ^ * ^ *S<'icntific ' 1 

i'v:rlii.illiinii % ' rcsea roll 

I _ ^1 

The value of such a scale is in realizing that we 
can improve our evaluations without becoming a"" 
scientisjt. Other locations can also be descnbecj^ 
on the/scale between the two extremes,''as foflowS: 

I Casual pveryiby J Rt»lf-cliccklnc I .Pn-it-ymirsrlf I Kxtni.slna V Sru'iiUfk- .1 
9 puihiatlims i mlii.itinns 1 fvaliiatitms | stiulk's | . x}';'*'^f*'l' 

The above locations are areas of the scale with 
no sharp lines of division. One b]^nd*s into the 
^ other. The first tjiree locations may be flescribed 
"as informal evaluatians^ which extension agen,ts can 
; do thfemse^ves..' , " >^ ' . 

Casual everyday evaluations need no further ex- 
. planation. They are like the first irhpressiDns of 
County. Agent Brown about his meeting or the um- 
brella decision. They are the ones we ordinarily 
make without mucitTHHisideration 'af the principles 
of eyaluatioi^in the declsio;is we make about simple . 
problems. " y . ' . ' 

Self-checking. evaluations represent a conscious' 
^attempt to apply principles of evaluation^ Mr.* 
Brown did some analytical thinking as he drove*" 
home from the meeting. >He was doing sotne eval- 
^ nation at a higher level He was jajsing questions 
^to himself about his initial feeling ^f satisfaction. 



He coiLsidei-ccI lo some extent the information he, 
obtained by hih: observalion.s and what conctiisipns 
•he eoLild draw Ironi those obsory^itions. - . 

vSeK-chcckin^i includes further checking on our ^ 
ordinary observations. It includes ^talking fuVfher 
* with otiieVs, writing to others for their jud^inont, 
A. sending mit a brief ftaestionnairc: having olie* filled 
out-^at a meeting and so'on. Mr. Biwn could da a 
^ . little checking as-fte went around the countiy to.see 
if any of the farmers had their .i;oirtested since the 
meeting.* » , , . '* . ^ 

Do-it-yourself evaluatioos involve more planning ' 
and* applieaiion of th6 i;i'incipl-es. of evaluation. 
■'I'hey are rtipre systematically done, more carefully 
planned and usu^ally require some technical help. 
^Each step in the evaluation' i.s considered/planned 
ancf carried out with due consideration to evalua- 
tion principles. ' ' j/ ■ ' * . ; 

These evali/fttions 'ai^ 'not complex ajid involved. 
,Thcy arc usually' surveys which produca usable 
results and which can, be done easily with some . 
training in evaluation or with some teehiiical ^help, 
, The best of these sometimes pass as data -for a 
master's thgsis. They blend into^the next loeatron 
•gjOn the scale, ^'Extension studies."* , ^ 

The objective of a three-week summer school 
, course in evaluation is tha'l the partidpants reach \ 
the location.on the^scale of a do-it-ypurself ev^ua-. 

■ ,tion; th^t they, plan and describe thcsteps in an 
evaluation whicl\, they can-do themselves u^pnj 
returning home. . , ' ^ - * ' 

The,Jast tw^o locatipns on the scale, "Extension* 

■ .studies" and ^'Scientific research." may be described 

as formal studies. - . . ' , 

Extension studies are more involved and com- 
• pjicated to pMan and carry out than any ojp -the 
» - pi-eceding locatfens on the scajp. They are broader 
y ^irt scope. They require greater attentipn to sound 
f principles oi scientific procedure in order to secure 
- theiicairacy needpd.* Theses for master's degrees 
usually fall in thi» location. 

Scientific research, of course, is at the "top" of 
the senile,' iijvolving very comp^ex. problems and, 
techhiques for getting information *fromA\:hich con- 
^ elusions can be drawn.. Long-time and experimental 
• -studies to deterinine caii^e and effect rehitionships 

'are chai-actcrisl-ic of this location. The invehtion ' . 
^ of complicated techniques and •proceckh-es-^'foi: get- 
ting* information (for doing research) falls -in this 
..locatiion. The top of th^i locati*on piqy illustrated 
by atomic ('cs,earcX satellite rese&Vch or cancer 
' reseaueh. , - . 

The scale of degrees of evaluation -servos as a 
) tommunications device iri a discussion for identiify- 
ing the kind -of^nquiry being considered and to be 
planned. It shows whether" the inquiry is' a very- , 
informal study or a Very, formal study. The kind 



of ''research'* under eonsideratian can be identified 
on the conlinuutn aild .misunderstanding cleared 
"up:^ The design and techniques .can then be adapted 
Jo what ^i.s valid and' pr^ictical in, the situation to 
"obtain ljv6 accuracy "noedecl*. / • ' ^' 

y • 

■■ . • * .' * 

Evaluation for Everydne 

;^ . Exten.\ion workers have feieen hearing abo,ut 
evaluatioi> during most^^ of ' their association with 
Exlens^ion. 'They sometimes- feel tjaat asJ'doers" 6/ 
extcns work; evaluation ancl research ar^.out of 
Iheir field; that evaluation' and research are some- 
thing 'mystical'or for persons with years of special- 
•ized training; that as extension teachers they have 
nevor done* and will never, do any ev^luation'%r 
..reseai'ch. - / - , ■ 

^The pontihuum. of deg^'ees. of evaluation ^hbws 
that there is a place 'for eyrfluationnn .evei'y exten- 
sion activity. Pi;^ictically anyone can do evaluation 
at tfin first t\VQ levels with no help at.alL Most of 
theni^can handle a dd-it-yourself evaluation with a 
little counseling and they can cooperate at the., 
otfier twp- levels. ' * ■ * * ** 
The-cbntinuum also sho\ys,tha,t we all commonly 
;^use ;nethods of/inquiry to get'information as a basis 
for forniing a- judgment, making a decision, or draw- 
ing a cdnclusioji." We can^ movejup the"scate and 
do better evalulitions, biit need, not beconvp fu}U 
tiqic Wsearchers at. the far end of the* scale. " 
vEafly in the days of the cooperation of the land- 
■ -grantdcqlleges ^and the U. S. Department of^iAgrr- 
culture in Msseminating' information relating to 
agriculture and home^ economics, it became evident 
. that if ^extension work were .to be* effective facts> 
w^ould be 'needed^ about its clrentele; personnel 
pi'bgram planniu'g, teaching nvethods, locaMeader- 
ship and other. Jactoi;^. ; 

-The Cooperative Extension Service began to 
carry ox\ studjes which' vvould provide information 
for making decisions about the conductof extension 
'teaching. ThcJse studies were more scientific than 
casual, everyday evakiations.* and broader in scope. 

It w^s recognized that since th^ E^ension Service 
was 'recSmmQiiding research findings to improve 
'agriculture; and -liomemaking, it must ''practice 
. what it preached'' and study itself through scientific 
-procedures in improving its work It was also rec- 
ognized thaf if Extension carries, on the study itself, 
4he findings will be better understoo'd and more- 
Tikel'y to be used. . 

Any institution should plow back into its work a 
percentage of its funds to-improve itself and, make 
its \\ >rk 'more effective.- This is* the function-of 
evaluation in Extension. 

As thcf^'size ami Importance of extension educa- | 
tion has grown, more "^objective information was 
needed. More formal studies have been made and 



more porsoimol }v v<.^ been assigned -to help exteiv 
.si{)n agents to nuike better casual everyday evaliia- ^ 
tio!j> and to make "(lo-it-yoinself * evaluations of 
their teaching. \ 

Evaluaiions Can Be improved 

rasual evcM-yday evaluations can be misleading, 
and often arc. because of errors in observation and 
inte'-pretation. There can be considerable differ* 
ence between what we see and what we thnik (intei - 
prct) we see. Gieat caiC must be taken to distin- 
.quish between what . we actually see and what we 
"read into" what we see. 

When We listen, for e.vanple. to a person speak- 
\ii<f — and espeoially when we road a printed page— ' 
i^tuch of what we seem to hear or see is really sup- 
by our n^Miiory or what we expect to see oi* 
hoar. Wo overlook misprints, assuming that wo 
.see the rij^lit letters, even though the wrong letters 
•are actually before us. In Vul^ paragraph two mis- 
[)rints have been made in.:entionally. Did you 
notice them'.' 

Our minds and our biases canjnfluence ourMnter- 
pretations and cause errors. thi?roby affecting our 
judgments, decisions and conclusions. 

Time and again it has-been shown that persons 
having fleeting glimpses of behavior 6r during the / 
oxcitment of the moment have made sworn state- ( 
ments of what they saw o?*ti^arfl only to have been 
[iroven positively wrong by further evidence. 

Casual everyday evaluation- can be improved by 
trying to avoid these errors. Some of llK? ways we 
can fortify ourseives and improve our evaluations 
are as follows: 

1. I.)evelop an attitude of mind by which we de- 
liberately try to provp jour -interpretation 
y.rong. Be critical of o|ir own interpreta- 
tions, 

2. Kecogni/.e our individual subjective bias and 
take account of it in our interpretatign. 
Before drawing conclusions from a single 
instance, make furthei- observations to deter., 
mine ho\\/wide.spread or consistent the be- 
havior is. 

'4. Check further to determine whether the be- 
haviyr was due simply to courtesy, to make 
;m impression, or was forced upon the per- 
son. Make allowances for or discount these 
things in our eyaluation. 

n. Consider other causes .that could have been 
responsible for what we observed and check 
on these causes. More than one thing can 
cause an effect. 

6. Be careful not to read into the observation 
what we hoped to find (wishful thinking) and 
overlook other reasonable interpretations. 



7. In evaluating a result or product of L-^havior 
use ci'itei-ia by which to judge the* result or 
• i)i-oduct, such as a score card. 
Check fui'ther to determij,e whether a prod- 
uct made or things claimed to have been 
done by the person were really dcme by the 
person and not by someone elso. 

9. Recognize tlwt many kinds of behavior are 
on a continuum, shafiing from one end of the 
scale to'another. and are not on an all-oi*-none 
basis. 

10. Determine whether the evidence we ob.served 
is actually -Ihe kind of evidence we would 
^ accept, or whethei* it cannot really be coiV 
sidered as evidence. 

IL Finally. :civoid jmnping to conclHsions. but if 
we nuist act (and that we muki do many 
times) before all the evidence is iii. act upon 
the best' evidence available depending upon 
the seriousness iof the consequence of our 
.actions. 



Tolerable Error 

('By following the preceding suggestions we can 
improve our evaluations. It would be unfortunate, 
however, if \*e go so far as to become perfectionists. 
Then we woui(1 not do any evaluation because^^ill 
evaluations p^ossess some error, . Human beings are 
not infallibfe and cannot follow the preceding prac- 
tices with' perfection. Furthermore, absolute per- 
fection is not ne.cessary. and not practical. 

Of course, in coljecting information upon which 
to base a decision, make a judgment or draw a con- 
clusion, it is necessary/ to collect ^'accurate" infor- 
mation and to make an ^'accurate" interpretation. 
But how accurate is **accurate''? No evaluation is 
{Kirfectiy accurate. ' . 

Casiral evQryday evaluations, however, are ctten 
subject to a high degree of inaccuracy, PyJmprov- 
ing our evaluations we reduce the size of the errors, 
at least to the point where we can tolerate the errpr 
that remains so we can make more axjcurate deci- 
sions. ' ■ V 

In measuring the length of a room, a tape meas- 
ure will give sufficient accuracy for purposes of 
laying carpet, or perh;^ps using a yardstick will be 
accurate enough. For some purposes merely **step- 
ping /)fC' the room is ac^ irate qnough. 

A show of hands at luCQtings of dairy farmers 
may be, accurate enough for some purposes to find 
out if the farmers are improving their pastures this- 
year. For other purposes you may need a carefur 
sample siu'vey which would show how they are 
improving their pastures. 

Suffice to say at this point that evaluation always 
involves some error but that better evaluations will 
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, reduce the error to a point where it is iiisignificant 
for practical use and will not mislead us nor others 
in makiii^ti decisions. Tonning judgments and draw- 
ing conckisions. . • 

Summary 

The general conce{)t of evaluation as a process 
of il) collecting information, and i2) applying stan- 
dards or criteria in (3) drawing conclusions', form- 
".K^ judgments, ,or making decisions, is useful 
because it can be applied to all kinds of situations 
confronting exteiision workers. All Kinds of pro- 
grams, methods, activities' and situations in Exten- 
sion can be evaluated. 

We do not need lo be research workers in order 
to use principles of evaluation. These principles 
can be applied by all persons. Perfect accuracy is 
not necessary; nor is it attainable, even in scientific 
rosearcJi. 



There is a wide range in degrees of evaluations 
from casual everyday evaluations' to scientific re- 
search as indicated by the continuum. 

We often make gross errors in our everyday 
evaluations and **jump" to conclusions. Care .in 
the use of principles of evaluation can improve our 
judgments and decisions. 

How far to go in applying evaluation principles 
depends upon the amount of error we can tolerate 
and still have useful results for our purposes. 

As will be discussed in the next chapter, evalua- 
tion is not an isolated process. It is a continuing 
process used at all points on the extension cycle. 
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^Chapter II 

EVALUATION IN EXTENSION 
EDUCATION 



J, XcU Raffddbaufjh 



What Is Cooperative Extension Work? 



'I he ('oopcM*ative Extension Service is a cienio- 
. cratic educNifional arrangement among the people 
in the counties, the State| land-grant colleges and 
universities, and the U. S; Department of Agricul- 
ture. It is legally reforrecl to as Cooperative Exten- 
sion Work in the basic legislation, Ihe Smith-Lever 
Act. which was passed bv C*giigress in 1914. 

The purpose of (Cooperative Extension Work, as 
slated in the original Act. - is " . . . to* ai'd in diffus- 
ing among Ihe people of t1ie United States useful 
and practical information on subjects-relating to ag- 
riculture and honie economics and^^^o encourage the 
application of the same." This Act further stipu- 
lates that "... agricultural extension work shall 
consist of the giving of instruction *and practical 
demonstrations in agriculture and home economics 
to persons not attending or resident in said colleges 
in the several conuTiunitics." and imparting to such 
persons information on said subjects through field 
demonstration. QubUcation and otherwise.'* it is 
within the framework of this* legislation, and other 
acts related lo education, that the objectives and 
.scope of cooperative extension educational pro- 
•. lira ms are determined. 

The basic administrative unit of the Cooperative 
Extensicm Service is the county. It is here that edu- 
cational programs are planned and where extension 
teaching is done. The basic objective of-the Coop- 
erative Extension Service is to develop the people 
into understanding, 'effective, self-confident mem- 
bers of society who will have broad vision and be 
capable of meeting responsibilities in their homes, 
their communities, their State, and their nation. 
We in Extension express this objective in terms of 
developing people to the point where they, through 
their own initiative and ability, may effoGtively 
identify and solve the various problems directly^ 
affecting their welfare. 

Some of Ihe major fields of educational jespon- 
sibility of the Cooperative Extension Service, in ad- 
dition to efficient agricultural production and ad- 
justment and home economies as such, are: Leader- 
ship development; conimunily improvement; farm 
ancl home nianagement; public*- affairs; marketing 
:md distribution; conservation of natural resources; 



housing; health; and social relationships, adjust- 
ment:! and cultural values.*' / 



Basic Philosophy of Extension Education 

In cooperative extension' education, it is ackhbwl- 
edgcd that people must be reached where they are; 
that is, ai their present stage of educational devel- 
opment and experience and at th(^ir present level 
of interest and understanding. It is the function 
of Extension to teach people hJvv to assess their 
own needs and how to solve their own probleins, to 
help them acquire^ knowledge and understanding, 
and to inspire them to action. The extension teach- 
er serves people by teaching them how lo help them- 
selves: in so doing, he teaches people, not subject 
matter. He teaches people how to think, not what 
to think. 

The success oS extension teaching depends on 
much more than knowledge and understanding of 
technical agriculture and/or home. economics. It is 
equally important that the teaqhers understand 
people and th^ir problems and that they know how 
to apply sound principles ot-t3ducation, psychblogy 
and sociology. 

The goal of the extension teacher is to teach not 
only desirable practices and understandings, but 
also the interdependence of practices and under- 
» standings so fundamental to achieving' satisfying 
family and community life. This is accomplished 
through programs which the people and extension 
personnel work out together. 

It is the f;'xtension teacher*s role, first, to inspire 
people, as individuals, families and communities to 
work together to identify their own problems; 
seconcl) to help them determirt^ their dbjectiv.es; 
and third, to counsel and supply\ technical help to 
them as needed in working toward these o^^tives. 

Teaching-Learning Process 

A brief consideration of education, and the ele- 
ments in the processes of ■ arning and teaching 
basic to the conduct of extension work, is helpful 
in undei;standing Extension as an educational pro- 
ce,s.' . as well as the place of evaluation in extension 
teaching. A definition of education which has been 
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atcopted generally by txten.sion vvorkcr.s and which 
ha.s beeit quoted often in extension publications is: 
"Kdueation is the production of changes in human 
behavior.""/ It is the aim of extensions-education 
to influence people to make desirable changes in 
their' behavior that will contribute to better farm- 
ing and honiemaking. to better family and com- 
munity living. ^ 

The fHlucatioi\id changes in behavior which con-' 
cerq extension teachers, and which are expressed 
in teaching level objectives, may be classified as 
follows: ' 

^ 1. Interests ^ ^■ 

2. ld*-Nds and purposes • " . 

A Information, k?lowledge and understanding 
4. Abilities. skilli5. habits and practices 
• i). Attitudes ^^nd emol^ rnal responses 

The steps in the learning process which contri- 
bute to the production <>f these behavioral changes 
have been identified and studied carefully^ Beal 
and Bohlen, in their treatuieni of the diffusion-pro- 
cess, suggest four steps which can be sunnnarized 
in this manner:- 

1. Awareness — People find out that the idea- or 
practice exists. 

2. Interest — People become concerned and rec- 
ognize that the idea or practice has merit, 

3. . Evaluation and Decision— People make mental 
applications of the'- idea or practice to their 
o(vn situations, arid rqject it or decide to try it. 

4. Testing and ^Adoption — People try -tfie idea or 
' practice and adopt it if .they are, satisfied. 

Extension. Lducation Evaluation 

Effective educational progranis-provide for these 
steps in the learning or 'diffusion process and con- 
tribute to production of the previously mentioned • 
kinds of behavioral changes. To evaluate a pro- 
gram, evidence must be collected about these kinds' 
of behavior and behavioral changes in the people 
concerned. ;EducationaI evaluation, then, is the 
process of determining the degree to which desired 
behavioral changes have taken place or are current- 
ly taking place as a result of educational effort. 
Similarly, extension education evaluation is the 
process of determining how well desired behavioral 
changes have taken place or are taking place as a 
result of extension educational effort. - 



These stages are fnvolved in su(X'e::;sful educational 
{)rograms regardless of wht^Vn^r' the educational 
approach used is intensive.: ;sndlyidual or family 
centered) or extensive Igroup oriented). Extension 
educators arc conccM'ned about. (1) the areas in 
which there is need for change, (2) what dii'cction 
thex'hange should'fake, (3) how rapidly the change 
takes place, (uul (4) how change can be directed so 
that desired accomplishments are maximizXid. 

There are many functions that nmst be perform- 
ed in the successful and efficient conception and 
, iniplementatian of action oriented educational pro- 
grams^. These functions can be organized logically 
into a sequence of actions from the inception of an 
idea^or the recognition of a need or problem, to the 
^1 final satisfaction-of Ihe need, or accomplishment of 
'the 'desired goal or objective. The steps which, 
usually occur in such a social action program can 
be organized into a logical tin>e sequence model to 
.provide a franiewoi^ for a planning and analyzing 
action oriented educational programs.- 

.'\11 educational change, just as other socialaction, 
, takes place \yithin the contest of social systems. 
The beginning of ail educational change is \vith 
convergence of interest and definition of need by 
a few people. This stage might be called the initia- 
tion stage of an action educational program.. 
- In most cpanmunities or counties there have been 
previous 'experiences .with similar educational 
action programs .which should be considered • in 
planning any new educatioriar effort. Seldom are 
all the social, sub-systems of a county involved in 
any one action program. This being true, it he- 
• comes necessary to outline the relevant social sys- 
tems, or the ''target system," for a particular educa- 
tional effort. Initiation of action of the ^'sounding 
board'* variety, and legitimation, or sanction giving, 
are important considerations in building^the foun- 
dation for any action educational program. " 

The process of organizing and planning for action 
in such a way that the educational need is defined 
and becomes, the felt need of the people, around 
which those in the relevant social systems commit 
, themselves to action and set objectives or goals for 
''themselves, is another broad area of concern in the 
conduct of action educational programs. At this 
stage, the specific series of learninfj experiences 
that will be needed to attain these objectives and 
goals can be set up in a plan of work. A plan de- 
veloped in this manner, and including evaluation 
for determining progress, can then be carried 
through. 



Extension Education and Social Action 

Through systematic analysis'of (he action pYocess 
of educational programs as they nurvc forward, 
-distiiu-t and related a^^tion stages are suggested. 



The Extensi>^ Teaching or Program. 
Development Cycle ' * 

In extension educational program development, 
five reasonably distinct stages arc recognized. Fig- 
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ure 1 and accompanying description "of each stage 
will help clarify this total prpcess and will further 
emphasize the importance of evaluation throughout 
each extension teaching or program dcivelopment 
cycle. 



2. DETERMINATION 
OF OBJECTIVES 



1, lOENTIFICATION 
OF PROBLEMS 



3. DEVELOPMENT OF 
PLAN OF WORK^ 



i. FOLLOW THROUGH 
OF PLAii OF WORK 



{ 




5. DETERMINATION 
OF PROGRESS 



EVALUATION 
DECISION 
PLANNING 
ACTION 



Fijgiire 1. — The Extension Teaching or Program 
Development Cycle 

^Stage L Identification of ^he Problem — ^Through 
collection of facts and analysis and interpretation 
of situations by the people working with, the exten- 
sion worker, the significant needs and interests of 
the people are identified. Those areas of needs and 
interests wherein it is believed improvement can be 
accomplished, and which lie within the scope of 
legislation applicable to the Cooperative Extension 
Serv-ice, then become the foundation for an inlmed- 
iate and /or long-range "program. This stage can 
be evaluated from the. standpoint of both content 
and processes oK procec^ures used. 

Stage 2. Determination of Objectives — After the 
needs and interests have been identified, the peo- 
ple, working with the help of extension personnel, 
can decide on their objectives. This involves a 
definftion of what is to be acconriplished in relation 
to the various needs and interests, both immediate 
and long-time, as well as how and by whom these 
accomplishments are to be instrumented. Criteria 
for this stage can be established and evaluations of 
both content of the objectives and procediires fol- 
lowed in their determination can be made. 

Stage 3. Development of a Plan of Work — After 
the needs and interests are identified and the^bb- 
jectives arc agreed upon, the plan to be followed 
for, putting the program into action must be devel- 
oped. Sound principles of social action must be 
followed in devejoping such a plan. Involved are 
such things as determining, (1) the specific jobs 
that need to be done, (2) the subject mattc^r that is 
needed, (3) the teaching techniques to be lised, (4) 



the activities to be undertaken, (5) the division, of 
responsibiliiies, and (6) the calendar to be followed. 
Evaluation based on established' criteria iot plans 
of work is very helpful at this stage of jtKe program 
cycle. • ■ - * 

Stage 4. Follow Through on the Plan of Worl^— 

A good plan of work presents the most effeotive 
way, considering existing circumstances, to ai.com- 
plish the agreed upon objectives. An actim pro- 
gram, based'on such a plan o/ work„and built on a 
foundation of sound education and social action 
principles, requires systematic and persistent effort 
from all concerned. Evaluations can be made of 
what is actually done in the action program from 
standpoints of both content and procedure. 

' Stage 5. Determination of Progress — The extent 
to which objectives are being accomplished is the 
basfs for determining how well a program has suc- 
ceeded. Evaluation of progress helps determine 
what remains to be done. When objectives are 
satisfactorily reached, new ones can be included 
in the program and plan of work for the following 
year. If the objectives are i)ot accomplished satis- 
factorily, a revised plan of attack on the same 
problem may have to be carried over into the new. 
program. In additiop to (|ata for future or followup 
programming, progress ^evaluation also provides 
tangible and objective data for use in annual and 
other periodic reports. 

Active participation of the people in all five 
stages is the core of the teaching-learning process 
in. Extension. ^ Each of these stages can provide 
many experiences for those involved and it is 
through such experiences that learning takes place. 

As indicated, each stage can be evaluated. This 
is important because the accomplishments and suc- 
cess6s attained in any one stage are indicative of 
how much is likely to be attained in each subse- 
quent stage of the extension teaching cycle. 



Summary 

In this chapter we have stated that the Coopera- 
tive Extension Service was brought into existence 
in 1914 to perform an educational function for per- 
sons not in residence at land-grant institutions in 
the fieliis of agriculture and home economics. 
I'hrougll the years it has developed an educational 
philosophy which calls for reaching and working 
with the people where they are in matters of educa- 
tional, occupational and socioeconor.iic maturity. 
Emphasis always has been on how rather than what 
to think. 

We have said that the teaching-learning process 
consists of producing desired changes in human 
behavior and that it is a dynamic or action oriented 



process.' Wo have said that evaluation is a* key 
concept in ino process and have outlined the fjvc 
stages of the extension teaching cycle with nota- 
tions as to how evaluation fits into each stage. 

Next we will go into greater detail as to 'what 
evaluation is and why it is so fundamental to all 
aspects of Extension. 
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Chabter III 

THE PLACE OF EVALUATION 
IN EXTENSION 

Joseph L: Mattheivs 



Our objectives in this chaj)tfer are' to help the 
reader to: (1) understand the role of evaluation in 
Extension, (2) know some of the. benefits that can 
be gained through systematic evaluation, (3) know 
what can be evaluated, and (4) feel that it is impor- 
tant to evaluate Extension work. While so doing, 
we hope to provide the reader with a usable answer 
to the question, **What is the place of evaluation 
in Extension?'' 

We evaluate so that we can have somie measure 
of the desirable, and undesirable, outcomes of edu- 
cational action that we take to move a person or 
group of persons towahd desired extension goals or 
objectives. We' are usif\g action here in the sense 
of attempting to chaiigo Luman behavior in de- 
sired ways. ' 

Why Evaluate? 

No doubt, this, question often has entered your 
mind. You may already have your answer, but we 
would like to discuss our's briefly. As we stated in 
the opening chapter, everyone does a certain 
amount of evaluation, sometimes consciously but 
quite often unconsciously. We ask ourselves, **How 
am I doing?'' at practically every stage of our work. 
, Every job we do (writing a news story, giving a talk, 
making a farm or home visit) brings up this ques- 
tion, r 1 

We may not frame the words mentally, i)ut after 
we have done something^we may feel good about it, 
or we may be only moderately satisfied with it, 
or our reaction may be one of dissatisfaction with 
what we did. ^By these reactions we have passed 
judgment on our activity and have found it good 
or bad. Making judgments in this ma^tlierabout 
the value of what we have done is a kiifa o; evalua- 
tion. , ' 

On a slightly^ higher plane is the kind of ^valu 
tion we do by assessinjg'wliat others say or c'.o about 
our work If a person who has attciided one of 
ypUr meetings tells ydu that he or she enjoyed the 
meeting, you feel good. If several persons tell you 
the meeting was a good one. you feel even better 
about it and think that it must haive been a pretty 
fine meeting. * j ' . _ . * , 

Expressions like these have some value as indi- 
cators of success. When nothing better can be 



done about evaluating our work, we come to depend 
a lot on this kind of evidence. However, w^e must 
not depend entirely dn such expressions from peo- 
ple w^ho participate in our extension activities. 
Quite often, persons making judgments of this sort 
have no real understanding of what the educational 
purposes of Extension are. For this reason, they 
have no sound basis^ for making judgments about 
results. * > 

Sound evaluation always -must be in terms of 
what we set out to do— the objectives we have in 
minde Therefore, expressions of the people involv^- 
ed, as well as all other measures, must be appraised 
in the light of what we expected to accomplish 
when we planned the educational activity!- 

I« addition to the danger that what people tell us 
may not be relevant to our purposes, there are 
other reasons why such evaluations must be supple- 
mented by evidence from other sources. Persons 
who have participated in our program and who are 
our friends are reluctant to be critical even\ when 
asked to be. This results from not wanting tjo hurt 
our feelings and from an inability to separate their 
feelings toward us from their objective appraisal 
of our profdssion2i\ activities. .Later in this manual 
we will have more to say about the complicating 
factors of the human element in scientific evalua- 
tion. ' , 

The preceding paragraphs lead logically to this 
statement:. "We all evaluate our work in some way 
so why n^l do it well?" If the first, part of the 
statement^ is not true, then experience would have 
value and there would be no improvement on 
tn&vjob as a result. of experience. Doing evaluation 
well Requires considerable self-discipline in apply- 
ing Scientific methods, collecting and analyzing 
facts, and making judgments about their implica- 
tions. This manual is all about doing '^'valuation 
well fay applying scientific methods to evaluation. 

What Can We. Gain from Evaluation? 

r *The list of benefits we can' gam from evaluation 
could include many more than the five important 
dncs that we will mention here. First, extension 
worker^ have to; make fendless decisions and then 
act ;:ccording to what they understand to be the 
"mandates of their decisions. ^ For example, a county 
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extension Vvorkcrn^^^ believe that appointing* the 
chairman of an actioa committee will result in bet- 
ter committee work than when the committee 
members choose their own chairman. Much later, 
and probably by accident, Jt may be discovered that 
the expected result actually did not occur. 

'The longer \a practice has been followed, the 
harder it is to be objective aboTut its limitations, and 
the harder it is to get at making needed changes. 
One reason for this is a predisposition to see the 
expected results even though they are not actually 
in evidence. Preconceptions about results often 
make perception of actual happenings so selective 
that unexpected or undesired ones go unnoticed. 
Evaluation is the mettiod of trying consciously and 
objectively to find out whether or. not certain 
activities actually did lead to the results that were 
anticipated.' " 

Good evaluation is essential when it comes to 
• making periodic qhecks on the effectiveness of 
educational programs to decide where improve- 
nients are needed.- These checks serve a function 
similar to the balance sheet in a business enterprise. 
When the sheet reports results that are -less than 
expected, owners or managers of the business look 
over the units which arc not doing well to see 
where changes are in order. In either case, deci- 
sions so basic to suc'cessful functioning of the 
organization must be founded on accurate measure- 
ments and observations, or changes may not be 
made in the best direction. Surely we all agree 
that change for change's sake is not good" long*run 
action, 

A third benefit of good evaluation is the feeling 
of security it can give the worker who is doing a 
good job. Extension workers can b :come so involv- 
.ed in their work that it is easy for them to worry 
about tlieir effectiveness and to have doubts as to 
whether they are really accomplishing their objec- 
tives. This concern is all right if it leads to careful 
appraisal and constructive* measures to improve 
programs. However, when systematic evaluations 
are not made, the tendency is for staff members to 
feel even less secure and to retreat to activities that 
have suVface indications of tangible results, al- 
though they may be less important to the work than 
others which they abandon. 

We may seek security by emphasizing extension 
activities to which people will respond in large num- 
bers rather tha'n others that could result in basic 
educational changes in the long run merely because 
we iTeed the immediate -assurance of favorable 
popular reaction. Scientifically done evaluation 
will give equally convincing assurance when good 
work is being' done and we know the criteria will be 
valid extension objectives rather than popular 
appeal. 



A fourth benefit lies in the area of public rela- 
tions. There is no better basis for sound public 
relations than a careful and. comprehensive evalua- 
tion. Rather than superficial impressions or value- 
clouded opinions, it can provide concrete evidence 
that can be adapted readily to the needs of people 
working in the various communication media. 

These folks are happy to help inform the com- 
munity of the Contribution of your educational 
program when they see the facts. They do not have 
the feeling they are ''going out on a limb" for you 
when they are handed the ammunition to deal with 
criticism. They know that critics can be converted 
to constructive Gooperators when concrete evidence 
of accomplishment can be produced. 

A final benefit we will discuss is the matter of 
what evalu'ation does to the ipvofe'^onal attitude 
of the worker who uses scientific e^^luaiion tech- 
niques. In the process of stating his objectives in 
systematic fashion, specifying the desired changes, 
analyzing the teaching activities designed to accoui- 
plish the objectives, and appraising the results and 
probable need for further action, he is very likely 
to adopt a more scientific approach in future under- 
takings. Inevitably, he will adopt more objective 
and systematic ways of thinking about his work and 
his accomplishments. 

Stages of Evaluation 

t 

We can think of evaluation as having two main 
stages. Early in the prograni we cr*n study arrange- 
ments and procedures for doing Extension work. 
Suppose we refer to this as means evaluation. Eval- 
uation of means is especially useful in the early 
stages of an activity or program before it- has been 
going long enough to accomplish the desired educa- 
tional results. We can study methods, or means, 
and decide whether they are worthy of the program 
.in which they are being used. * - 

Examples of means evaluation would ■ include 
appraisal of the organization of people for extension 
committees, clubs, spedi,al interest' or commodity 
groups and the like. Educational materials used 
such as bulletins, leaflets, newspaper articles, writ- 
ten programs, and plains of work are 'included 
among the means which should be evaluated^ The 
procedures for planning programs and. conducting 
club meetings also should be considered. 

Evifluation in terms of means alone is hardly ade- 
quate, however, and should be thought of as part 
of the evaluation sequence rather than the whole 
of evaluation. Insofar as the extension program is 
concerned, the end products of extension work, the 
educational changes that resjult from extension 
teaching, are what must eventually be evaluated. 
We nmst make ends evaluation. We need to assess 
the changes in knowledge, understanding, attitudes, 
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i!Is and .abilities of tjic people who are affected 
by Extension educational activities. 

Purposes of ^valuation 

In addition to kinds of evaluation in terms of 
stages of the prdgram, we can think of evaluation 
in terms of objectives. We can think of it ih ^errns 
of pure fact finding or measurement; or. we can 
think of it as a basis for interpretation. In the first 
is involved matters of assembling measurable data 
l^on changes presumed tO result from the teaching 
device — observable behavior. Ghanges as well as 
changes in material things. In the second we make 
considered judgments abouLthe data that were' 
collected by means of the measuring techniques. 

Although most of us can recall when we stopped 
at the first stage, or when we observed others stop 
there, we ^should have no difficulty in acknowledg- 
ing that the evaluation mission is not accomplished 
unless we face up to the interpretation stage. At 
some point judgment must be made about the' de- 
gree of success or lack of success that was exper^^ 
cnced in the program. 

Who* Should Do Evaluation? ^ . 

Evaluation, insofar as possible, should be done 
by the extehsion workers who are actually respon- 
sible for the program that is beipg evaluated. It 
improves their understanding of the program and 
their work in general. In using the metHods of 
science to evaluate their work, they accumulate, 
reliable evidence which helps them define future 
problems more sharply. They draw on these ex- 
periences and findings for action hypotheses that 
give promise of helping to reduce or eliminate diff i- . 
culties in their work. They test out promising 
procedures on the job and accumulate evidence of 
their effectiveness in the next evaluation sequence. 
Eventually; after obtaining sufficient evidence of 
effectiveness, they feel justified in generalizing 
within limits and can turn to other objectives. 

An important psychological consideration in 
evaluation is the fact that the people who stand 
the best chance of benefitting professionally from 
the experience of the evaluation are the ones who 
do the evaluation. They can see at first hand the 
effects of the methods and^procedures and can get 
speedy answers as to how good new and promising 
ways of dojng the job really are. For this reason 
persons responsible for the pro-am -should make 
every effort td do the evaluation or be a part of 
the evaluation team.* . - ' 

Another consideration in evaluation is the prob- 
lem of communicating results to the ones who can 
use them in tb > work. The communication prob- 
lem is solved p roportion to the degree of partici- 
pation by the "people, who are responsible fdr the 
program. *\Vhen the responsible persons are fully 4 



involved, or when tney take total responsit(ility for. 
evaluation, the communication problem virtually 
is eliminated. 

Sometimes, however, the complexity of the prob- 
*lem being studied or the technical competence 
needed, may make it impossible or impractical for 
the extension worker to conduct the evaluation 
• effort independently, Nevertheless, drawing upon 
^ the skills of others for assistance in evaluation. need 
not impair communication nor minimize the value 
of the experience to the extensiori worker whose 
work is being studied. Experience has shown that 
there are some things that extension workers have 
neither the time nor the technical competence to 
handle b}r themselves. ~ These, very properly, are 
left for State extension research persohs .and for 
the resident teaching or experiment station re- 
search staffs to deal with.^ 

, Summary 

In this chapter we have attempted to impress our 
readers with the im^^ortance of evaluating our wprk 
on the most scientific level the particular activity 
warrants. We have listed and elaborated on five 
important benefits to be gained by evaluating. We^ 
have mentioned that there is both means and ends 
. evaluation, and that *>oth Have a place in extension- 
teaching, but that \\~ must not stop after (doing the 
former when it is possible to do the latter as weil. 
We have stated also that evaluation can be a matter 
of .measurement alone, or it can be carried to its 
logical conclusion by interpreting results of the 
measurements to determine effectiveness of the 
action that was evaluated. \ - 

Closing remarks were devoted to explaining why 
extension persons responsible for the program 
either should be the ones to evaluate it, or should 
be as closely involved as possible in the process. 
At the same time,, they should feel free to call on 
professionally qualified counsel and leadership in 
situations calling for more facilities than are at 
their own dis^posal. 

In the next chapter we will spend some time 
discussing what is involved in scientific methodol- 
ogy and what is necessary, to consider before "doing 
a scientific evaluation. 

— . . ^ 
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THE SCIENTIFIC APPROACH AND 
EXTENSION EVALUATION 
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- People are curious. They have always- been cur 
ious. They. want to find out what makes things 
*'tick." They want to understand themselves, othec 
• persons and the w^orldjn which they Jive.- They try 
to study ancj investigate puzzling questions. • 

This curiosity is not confined to adults. At a 
very early age .people are curious. The *'why" 
questions of children are well-known to parents 
and teachers. 

^ People. are not only curious but they also have 
problems and must make decisions. Very early in 
the' history of man, the problem of measu/ ing time 
■was recognized' Man put his ingenuity to work to 
find a solution.. The measurement of time; pro- 
gressed o^ver the years from such gros^ unrts as 
thcee moons ago, through the hour-glass stage to 
electric clocks and'self-wihding watches. 

How do people find, out things? '* How- do tfiey 
make up their minds? How do they know what to 
. believe, fhd what -to do?' ^ . > ' 

During the history of man's develo]f)ment, people;,: 
have used a variety of approaches to gain- under- 
standing and to solve problems. Th^ey, have used 
' custom and tradition, aui .lority, personal Experi- 
ence and intuition,' and the scientific method. . * 

Custom 4nd Tradition 

We lise custom and tradition many times in our 
lives in reachin^^ decisions— and with no harmful 
affects, as«in bxijing clolhes, and many times for 
highly essential ard valuable purposes, as for ex- 
ample, courtesy anil manners. 

But when we use custom and tradition for all our 
decisions, improvement and progress -/go, out the 
window." If farming d.ecisions were dictated en- 
tirely by the farm practices of our ancestors/ exten- 
sion agents would have a difficult time implement- 
ing the findings of the experiment station, 
yAVe would have no automobile,^ no radio, no elec- 
/ric power/ n"o telephone and'>a host of>other things 
•/because they were not our custom or tradition. The 
past would be perfectly preserv*ed. 

What would happen if extension agents made all 
their decisions on the basis of custom and tradition? 



Authority 

Custom and tradition did not always, provide a 
satisfactory answer to curious people; nor solutions 
to urgent problems. People appealed to **authori- 
ties," those in whom they had confidence and whom 
they thought should '*khow." 

They went to the **wis6 men of the tribe," to the 
oracles, to the "medicine, men" and up through the 
ages to the .authorities of today. Much of our 
•teachin^g in schools/and colleges is accepted on 
authority. We cannot possibly explore and com- 
prehend all knowledge. So we depend^ on- author- 
ity. ^ 

PeopTe frequently consult technical and profes- 
sionar authorities like physicians. County extension 
agents, teachers, lawyers, bankers, and so' on. 

'^'ootball players are governed by the rules of the 
game. Referees make their decisions on the basis 
of the rules. Card players -are/governed by the 
rules of the game/ Extension workers are governed 
by the policies of the institution. 

But authority must get its knowledge in some 
way and arrive at the rules of the game, the policies 
of an institution, or the advice of experts. 

•Personal Experience anrflnturtion'-'^ 

Personal experience is a widely used basis for 
making decisions. Notice how often it is resorted 
to in argumentative situations, and hovy it is given 
as a reason for our decisions. This is done con- 
sciously. 

But intuition is so^what different. Intuitive 
insights seem' to come from nowhere. They areT 
hunches. Have you ever worked long and hard on 
a problem and have finally gone to bed without * 
solving it? Upon. awakening in the morniijg, the 
solution came to you like a flash. Where it came 
from is difficult to say. It just came. 

Or have you ever^had to make a judgment when 
you know what it should be but could give no good 
reason for it? These are two examples of intuition. 
Such decisions or judgments seiim to be "based on 
p-^rsonal experience and to have come out of the 
subconscious. 

' Long practical e^xperience m a fteid of work tends 
.to develop this intuitive power in Uc' and to make it 
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easy for us to use intuitive judgments. They are 
■ frequently right, and they are. frequently wroifg. 

Personal experience can be vei7 misleadihg at 
times. For examt)l€f, vve may draw a conclusion 
from a single instance^. Note in Chapter I othe'r^ 
common errors \vc^ frequently make during per- 
sonal experience in* getting information and form- 
ing judgments. , 

Following the rules of evaluation will help us to 
use personal experience as a morcsatisfactory basis 
for making decisions and checking on- our intuitive 
judgments. • 

Scientific Method 

/ Toward 'the end of the 16th century another basis 
for maknig^dfcgisions was tried. This was cheeking 
to see, testing it, experimenting with'-'^^it, trying it 

The often quoted illustration' of .the use of the 
scientific iiYethod was Galileo's experifnent. , In 
Italy in 158y Galileo and others were arguing, about 
whether a heavier weight^Us faster than a lighter 
weight. Some said that it stands to reason thai; a 
heavier weight will fall faster. 
; Galileo proposed that they drop two weights, one ' 
heavier than the otlier, [vom the leaning tower of 
Pi.sa at the same^time and sea which one hit the 
ground first. This was a*niost unusual way of de- 
.ciding a question at that time. 

Eventually Galileo tried this way. Both weights 
when dro{^ped at the same time reached the ground 
at the same time. They dropped with equal speed, • ^ 
neglecting air resistance. What an odd way to 
*'prove'' anything.' This was a revolution in think- 
ing. It opened the way for discovery. It freed the " 
minds of people to find out new things. 

Today we take Galileo's method -as an acceptable 
method — or do we? How many times do we make , 
decisions, form judgments, jump to conclusions 
without checking to obtain a better basis for our 
actions? 

The method used by Galileo is the method which 
has made it easier to understand ourselves and the 
world in which we live. It is the method which 
.has,made^possible the progress of tod?y. it is the 
method of evaluation. 

,;■ - 

Characteristics of the Scientific Method 

There are certain characteristics which identify 
the scientific method of making decisions, fbrming 
judgments or drawing-conclusions. We will discuss 
five of them briefly at this point and elaborate on 
them in succeeding chapters. 

1. Factual — The scientific 'i^i^ethod is a way of 
vv;orking by which observable and demonstrable evi- 
dences can be obtained 'and upon which people can 
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'agree.*" If we want to find out whether farmers are 
adopting certain practices or have certain attitudes, 
we can do as Galileo did — loQk and see, * 

,2. Analytical — Most oLine is very complex and 
involved. It must be broken down into its compo- 
nent parts so that the human .'mind can deal with*it. 
Most processes in Extension are too complex to ■ 
study as entireties. . , ^ 

Even leadership, a component part of extension 
education, is not manageable until it is analyzed to 
the point where there are '^handles" that we can 
take hold of and^ understand. The big intangible 
problem must bd broke^i up into slnqll more tan- 
gible problems. ' • ; 

Evaluation may seem to a very complex pro- 
cess. Through analysis it can be^broken'down into, 
its component aYiri'understandable parts. Later in 
this chapter steps in evaluation are presented and 
succeedin/^ chapters amplify these steps. • 

3. Impartial — Scientific method is 'a method of 

attacking problems. Scientific attitude is ^how^we* 
feel abo,ut using scientific method. It is a disposi- 
tion or wiilingfiess to apply scientffic thinking. 

A debater argues to prove his side of an issue. He ^' 
canot'have a scientific attitude. If he did he might 
weaken his argument, and lose his case. ,He dare 
not present both sides 5f the issue. The opposing 
debater presents the other side. 

A* lawyer, like a debater, cannot argue for both 
sides.' He supports and .strrngthens his "own side'- 
of the case only. 

A judge or jury, on the other hand, must have a 
scientific ^*titude. They inustMook at both sides . 
and weigh the pros and jops of evidencelmpartially. 

A scientific attitude is a willingness to looli,at our 
interests, prejudices, and loyalties. It i*3 a wjlling^ 
ness to examine and re-examine our behefs and 
explore new ideas. 

A scientific mind seeks what is just, true\and 
beautiful. It is alert, curious, interested and a little 
excite^l about the world. It^is an open mind, (not 
to be confused with an empty mind)* It is an atti- 
tude of, mind that is learned, riot inherited. 

4. Reliable— Have you ever'noticed that just be-. ^ 
fore Chtistmas children are on their good behavior? 
During the rest of the year they are just normal 
children. If you observed those children only just 
before Christmas or only at some other time of the 
year when their behavior was at low ebb, you 
would come to opposite conclusions. 

Reliability means that vve should have a sampla, 
of representative observations arid be careful about 
drawing conclusions from a sjngle obs^efvatior at 
any particular time. 

5. Objective — Each of us uses our . subjective 
judgment in interpreting what we see and in draw- 
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in^' conclusions about our observations. Do othcis 
.scf uiial \vc see and do they dra.w the same conclu- 
sions? The degree to which they, do so is objectivity.' 

If we look at dog, each of us would agree that 
it .s a dog. "Except perhaps at a dog sho'Wr '^^'^ might 
see in entry and ask, 'is that a dog?'' ♦ 

Objectivity is the agreement 'among people of. 
. what they observe and of their interpretations, : In 
evaluation we are- concerned about agreement on 
the evidence we are looking lor and upon recogniii- 
ing it when we find it. ^ 

We are cqncerjied alJout agreement and under- 
^> standing of the methods used in evaluation. /• We 
are concerned about agreement ait the interpreta- 
tion and conclusions reached.. " 

Majiy arguments result fi-oni differences of opin- 
•ion. .The use of scientific thinKing helps to dispel 
these heated arguments. W-hen people clarify what 
..they are arguing about, they often find that they 
are not arguing about the same thing. 

Have you ever been in a discussion where after 
an hour the group found that all were not talking 
about the same thing? Clarifying thC question 
makes 'for better communication^, more efficient 
■ use of time and better human relations. 

The ubpve examples illustrate the need of objec- 
tivity to assure understanding, of the question 'at 
issue. • . 

Objectivity- is inci'eased by applying numbers to 
our observations. Whereas *'a few^persons" may 
mean 3 or 10 persons. * '5 persons" is Very definite, 
"Five persons" i.s^more objective lhan '*a few per- 
sons." All o4* lis can look and^see 'if there are five 
persons. 

At One tjme length or distance was not expressed 
qucrtititatively. Sweh expressions as "a piece way 
clown the road'* or ' quite a long way from here*' 
wcye used. NO;W we measure distance in the num- 
ber of inches. feet» yards, miles and'even light-years. 

The thermometer is a more objective measure of 
h'oat than our feelings of being hot or cold. It has 
been said that p'rogress in scientific study has 
accelerated with the ^development of -satisfactory- 
quantitative measurements. These are used exten- 
sively at the upper end of the scale of degrees of 
evalur'-on, 

M. ..y things in the behavioral sciences today. - 
however, cannot be quantitatively* measured; they 
can only be described. As time goes on, and better 
measuring devices are built, more quantitative 
measurements will be made. 

Science and -Scientific Method 

To some people lesearch .is associated with sub- 
ject-matter fields.* Science is considered to be in 
the fields of chemistry and physics. To these per- 
sons the:l)ioh"gical fields such as medicine: plants, 
janimals and food are considered on the fringe of 



^science. They be/lieve the social studies like edu- 
cation, psychology, sociology, anthropology, ^and 
similar fields ai7e clearly outside the realm of 
science. / ' :• 

Other persons claim 'that all of these fields are 
in the realm ofjsciencc?. ' The difference of opinion 
raises the question, '*What is science?'* Is science 
a field of subject matter or is it a method of think- 
ing? To the latter group of people, phy.'iical, bia- 
logical, or/sociaT phenomena are not in themselves 
science^.imtil .the human mind ^brings to bear on 
these phenomena its .powers of observation, analy- 
sis, synthesis, and verification in organising these 
phenomena for use by people. 

In this sense, science as a method 'of thought has ., 
application to all fields of subject matter. Scientific 
method involves the logical thought processes based 
on the recognition of assumptions and definitions. 
It involves the collection and interpretation of data 
and the control of factors which might influence 
the lesult of an experiment, a study or an ihvesti- 
cgation. 

.. Mathematics can be applied to all the fields, more 
accurately to some than to others. In similar man-, 
ner experimental controls can be applied to. all the 
fields,' W'ith much more difficulty in some, than- in 
others. ' 

Thd past 50 years have shown marked progress, 
in the application of scientific method to problems 
in the field of the social sciences. Evaluation in 
human behavior has shared in this progress. Today 
we have evaluation in extension work as an appli- 
cation of scieatific method with various degrees of 
rigor depending "upon practical conditions and 
usefulness of. the results. 

Five Questions ^ 

We have found that by asking ourselves five 
simple questions we can get off to a fast start on a 
problem requiring evaluation. Slight modifications 
will make them fit most'situatioiis. 

1. What Do You Want to Know?— The first thing . 
you need to do is decide exactly what yo^i want to 
find out. What are the questions you want answer- 
ed by your evaluation? Work, out these questions 
as carefully as you can. because they determine 
anci guide the conduct of the evaluation, and be- 
cause the answers will help you in your work. 

The questions will guide you in stating the speci- 
fic questions for the questionnaire or other devices 
you use. Analysis can be made of the i^eplies you 
receive to these questions. The repoil of your 
evaluation can be organized around these ques- 
tions. In 7act the report will be the answers to 
these questions. 

By carefully determining what you want ^ ■ 



[15] 



ERLC 



^4 



V) 



know, an initial, vague, general fueling of concern 
can he broken down into questions which you can 
tackle. ^ * / . 

2. Where, and How Will You Get the Informa- 
tion?— This step has two parts; where to get the^ 
information, and how to get it. 
a • Where means ffom whom or from what other 
source. Will it be from adults; from men, from all 
farmers^ in a county, from dairy farmers, or from 
Gxade A milk dairy farmer's? Will it be from' all 
persons in any one of the groups or from a sample 
of persons in one ofthe groups? 

Otiicr sources of information a're - records and 
reports, such as annuM'i'eports, the census, written 
programs, minutes of meetings, and so on. 

How you will^get the information refers to mail 
ciucslionnaircs, ' observations, interviews, tests, 
check list.s. scone cards, case studies, and so on. 
Forms arQ prepared on which the information can 
be recorded. They are used to facilitate handling 
the data and so you do not have to -depend upon 
, n^emory. 

When you have thought through and know the 
infoi'mation you want and also where and how^you 
will get ft, you are- well on your way in planning 
an evaluation. 

^3. Who Will Collect the Information?— The next 
step h to decide.^ whether you or a group of other 
i persons will collect the information. What training 
(Avill be necessary if you, have, others collect the 

iinformation? 

1 

^ 4. How Will the Information Be Analyzed? — Now 

is the time, to go back again to the tivst step — the 
information you wanted. The replies on the record 
forms are organized and summarized to answer the 
questions formulated in step one. 

5/ What Does It" Mean? — What are applications 
of the results to extension work? "In what way can 
they be used to make our work better*^ After know- 
ing these results, what changes should one make 
in his wayiof doing extension work? 

Ofti . the changes are not big revolutionary 
-changes. Many times there are little ways, such as 
. doing something different in organizing a group, 
or in writing-a letter, or in a farm or home visit. 
You may get to know your group better and under- 
stand why some dp not attend meetings or why- 
they arc not interested in new practices. 

All of these steps are thoroughly discussed in 
succeeding chapters. - * 

An Evaluation Outline 

The fiv.e questions we have been rffecussing can 
be expanded into an evaluation outline as a plan of 



action that yqli can use in makirfg "do-it-yourself 
evaluations" and "extensi6n studies," ■ ^ 

'Although the steps have a certain element of 
sequence to them, it is wise to keep all steps pretty 
well in mind while working out the details of any 
one of them. All the steps are not usually taken in 
any given order. Never consider only one step at 
a time without thinking about the other steps, since, 
all steps are related to each other. 



1. Need for the Evaluation 

a. What extension project, problem,, activity, 
job, method or situation do you want to 
evaluate? 

b. Why'evaluate it? 

' c. How can the results of thevevaluation be 
used in your extension work? 

2. Purposes of the Evaluation 

a. Is the evaluation, 

(1) an analysis of a situation, or 

(2) an evaluation of teaching objectives? * 

b, What questions should be answered by the 
* evaluation?"^ 

3. Questions to be Answered by the Evaluation 

^ a. If analysis of a situation, clarify the kinds 
of information needed to answer the ques- 
tions. 

b. If an evaluation of teaching, ^ clarify the 
teaching objectives. Analyze the leachmg 
plan. 

(1) Review what has been taught. 

(2) How it was taught, and to whom, 

,,(3) Changes in behavior expected as a re- 
result of the teaching. ^ 

4. Sources of the Information 

a. People — farmers, homemakers, local lead- 
ers, club members, non-members, extension 

, agents, etc. 

(1) Do you need a sample of these people? 

(2) How will you draw a sample? 

b. Recorded information — reports, census, 
etc. 

*' 

'5. Collecting the Information 

a. How? Interview^, mailed questionnaires, 
observations,, etc. 

b. By whom? Person making the evaluation, 
extension personnel, local leaders, etc 

6. Selecting or Constructing a Record, Form 

a. Kind of form; questionnaires, interview 
' forms, tests, ^ observation sheets, rating 
scales, check lists, score cards, anecdotal 
records, etc., to be used. 
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b. Data about_ -^^^ 



(] : tHe situation to be studied, 

evidence of progress toward the teach- 
ing objectives, or ' . 
(3) **face dota^^ to be cdlected. 

c. "Formulation of questions or statements 

d. Physical set-lip of the record^fdrm 
c. Pretesting and revisions 

7. Analysis and Tabulation of the Data for Use 

a. Classification and sorting of data to answer 
questions in Step 2 b. 

b. Work tables needed " 

c. Coding of free-answer questions 

d. Method of tabulation, 

c. Sorts and sirbsorts to bring out relation- 
ships " . .. ^ 

Reporting arid Applying the 
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Ititerpreting, 
Findings 

a. Preparation of tables, charts, and graphs 

b. Sumniai-y of fi".dings 

c. Applications to extension work. . 



Summary ^ " 

The curiosity and necessity of people to learn 
about themselves rnd their, envii'onment has occa- 
sioned them to try various methods o^ getting infor- 
mation and making decisions. ^ 

The most satisfactory method has been the 



scientific method. It is the method of careful obser-' 
vation and control of subjective bias. Scientific 
method can be used in solving all kinds of problems 
and is not limited to the physical sciences. 

Five simple questions in evaluation are: 

1. What information do you want to get? 
,2 / Where, and how will you get the information? 
3. Who will collect the information? 
, 4. How will it be analyzed? 

5. What does it mean? • 

These questions were expanded into an evalua- 
tion outline. . 

The scientific method is a method of thinking 
and we can apply it in our everyday work. 

The four chapters of this introductory section 
are designed to explain evaluation, its relatior^ to 
extension education, how it is used in extension 
work and the scientific approach to the study of 
problems. / \. - 

In Section II, methods used in the evaluation pro- 
cess will be discussed in detaiU 
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In a more comprehensive text we yiight be justi- 
fied in devoting a chapter to-outhning the shades 
of difference among concepts such as purpose, ob- 
jective*; intention, goal, need, aim, drive, want, and 
others. For some pifrposes, we in Extension do 
make certain distinctions amopg them. For im- 
mediate purposes, Jiowever. we will^se them inter- 
changeably, but will tend to use "goal'' or "objec- 
tive" most frequently. 

i^eed for Objectives 

Mow often have you heard renvM"ks ' like this'? 
';Everyone knows what we are ti'^ing to accom- 
plish; let's get on with the job.-' Statements of this 
kind are not uncommon among people working with 
Extension educational programs. \'ery often the 
importance of spending the time necessary to deter- 
mTne and define goals is not realized by laymen, 
and all too often professionals who work- regularly 
in the field of extension education make the same 
error. Because of this, we may find "professional 
educators, lay leaders and. educational groups so 
concerned witlj-their on-going procedures and activ- 
ities that they lose sight of what it is they are trying 
to accomplish. They fail to realize'that educational 
objectives are the criteria by which content is out- 
lined, materials are selectedj^ teaching procedures 
and learning experiences ai^ developed: and pro- 
gres.s toward accomplishmei^t — and accomplish- 
ment—arc evaluated. ! 



Chapter V 
GOALS AND OBJECTIVES 

./. Neil Raudabangh 



Kinds of Objectives 



In considering goals or objectives in Extension, 
it is essential that we think of them in relation to: 
the people involved, or to be involved. Some ob- 
jectives will be group objectives and some will be 
individual or family objectives. Once goals and ob- 
jectives are determined, they exert a definite influ- 
ence on the activities and behavior of the people 
participating, as well as on' tht extension Worker 
involved. Time spent by extension agents helping 
people to define their goals and objectives can do 
much to help them gain that needed direction and 
purpose which will, in turn, motivate them to in- 
creased achievements and satisfactions. 

Goals and objectives indicate what people are 
trying'to accomplish; they identify the target to- 
ward which the action program is oriented. They 



can be long-time or immediate, they can bo general 
or specific. In an educational program it is desir- 
able to\(listinguish between long-time (ultimate) 
and short-run (immediate) objectives. Both have 
their place in extension teaching, but ii is important 
to the total program that they be compatible. 

In any consideration of objectives it is also essen- 
tial to distinguish between brOad objectives and 
specific objectiv:3s. The latter are more apt to be 
achievable, dynamic and capable of doing much to 
help promote action and participation. The former 
are achieved with greater difficulty, largely because 
progress is not so apparent because of the time 
span, and because agreement on feasibility is not 
so universal. ' ' , 

People working with action educational programs 
sometimes adopt a set of abstract objectives and 
then proceed to consider them as though they are 
short-run and easily achievable. Such objectives 
especially for extension educational work, fail to 
provide needed direction. They limit prospects for 
achievement and ultimate success of a program, so 
necessary to stimulate in^he people concerned the 
desire ta strive for growth and development. 
.^Pjy^fHe, both individually and in groups, respond 
bCist^to specific objectives that they believe are 
achievable. If educational programs are intended 
to stimulate people to participate, dynamic and 
achievable objectives are requisites. Such objec- 
tives also serve well as the basis for identifying 
evidence of accomplishment for purposes of evalua- 
tion. 

Levels of Objectives' 

The relationships and differences between objec- 
tives of extension workers (teachers) and those of 
their clients (learners) are not always realised. This 
fdilure results in much ineffective teaching, and 
contributes to the development of unfavorable 
attitudes toward educational programs. 

Extension workers, who ^re not sure what the 
people's 'Objectives really are, have to conduct an 
educational program they believe is needed, present 
subject matter they consider appropriate for the 
program, and comfort .themselves v*^ith the hope 
they are achieving desired objectives. Actually, 
they'may be producing few, if any, changes in the 
people they are attempting'to help. In cases of this 
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kind, the peopFe thciiiselvus, wlule failing to iiiulc»r- 
stand aiul accept the objectives suggested by the 
extension uorker. may have no idea of theii* own 
objectives. 

When dealing with this frequent conflict in ' 
objectives, it has been found lieli)ful to think of 
objectives on various levels, each with a i}lace and 
function. Burton, in his book. 'The Guidance of 
Learning Activities/' has classified objectives into 

bfour levels. ' 
^ 1. Objectives of Society — The general aihi of 
Mfvhvy society is to secure Ihc 'gbod*' hfe for all of 
its nienibers. Surely all of us have spoken glibly at 
one time or another of high standard of living, 
niaxiniuin i)roducti(.ri. favorable balance of trade, 
religious freedom, and such generalities. Remote 
and abstract objectives of this kind have a place in 
designing all-inclusive ultimate ends, but Ihovrajie 
of little value as objectives for an action oriented 
exlensitjn educational progiani. 

2 Objectives of a Specific Organization or Group 

» - "Hiis isvthe level of objectives such as are stated 
in the printed materials rcdative to State and federal 
activities of the Cooperative Extension Service. The 
• objective toward wlVh^h^hese agencies are directed 
is impi-oyement of the ec0IT^mic. social and spiritual 
well being of farm families in general. Objectives 
at this level have the large gi'oup rathei- than indi- 
viduals or families in nniul and, for this reason, are 
not suitable for designing action extension educa- 
tional pi'ograms below the State level. 

•I Objectives of the Extension Agent — An ex- 

aniple of an objective at this level might be to 
develop in livestock farmei's an understanding of 
the merits -of stilbesterol in livestock feeding. A 
broader one at the same level would be to intci est 
l)arenls of junior and senior high school children 
in ways of improving their understanding of teen- 
agei-s. 

Characteristics of objectives at this level are that 
they consist of what changes agents desii'e to bring 
about in the |)eoplc with whom they work. They 
serve as evidence of the ability of the agent to 
translate extension objectives into action objectives. 
Flowevei', agents who work exclusively with objec- 
tives appropi iafo to this level can never be positive 
how well their plans fit the fieeds of tlieir particular 
clients. 

4. Objectives of the Clientele — These are the 
things the people wish to do. or accomplish. A 
fanner may desire to increase his income by ten_ 
percent by improving his cattle feeding operations; 
a homemaker may want to nuike a coat or dress in 
order to reduce the social stigma of an inadequate 
wardrobe and still keep within the family budget. 

An extension workei* cannot always be aware of 
people's purposes or objectives. He may have to 
- . ^ 

' ^ . ' I 19 

ERLC 



set out deliberately to leai'n wliat they are. How- 
evei*. an alert worker can spot some of Ihem as they 
emerge in the course of routine activities. MIe must 
think throiigh situatious and attempt to foresee 
what si)ccitic desires or needs are likely to reach 
the awai-eiife!ss level of the people. Often it is nec- 
essary, and desirable, for him to do a little stage 
setting and manipulate the environment to help the 
needs and ])urp()ses emeige to be recognized. 

These four levels of otjjectives are not related to 
each other in a sti'ictly additive manner, but they 
cannot be entirely iiiconipatible with each othci*. 
The objectives oX :uiy teaching^learning situation 
in Extension must oi'iginate in the situation, itself. 
They should be dei'ived from factors present in the 
situation which are of" concern to the people. 

The iinmediatc objectives of the people are the 
pi-ogress objectives. Ttiese pi'ogi'ess objectives 
bear upon their moi'e remote aims. The extension 
agent, as teacher and educator, must exercise lead- 
ership and guidance to see that a balance in cni- 
.phasis between itnmediatc and ultimate objectives 
is achieved. 

Those most directly concerned often do not i-eal- 
ize the need for this balance. Agent attempts at 
developing awareness of this need can be cohfusing, 
and have been known to antagonize the clientele. 
Objectives at any level need to be consistent and 
compatible with those at other levels, just as 
fnunediate or short-run objectives need to be com- 
patible with ultimate or long-time objectives. 

Objectives of the extension worker and his clients 
need not be similar in form but they need to have 
a coiumoii core. The people's objectives arc those 
which they believe can be accomplished through 
activities in problem solving projects of their own 
design. The extension woiker's objectives arc the 
desired educational outcomes which ho hopes the 
people^ will achieve. 

The extension woikei-'s role is to organize and 
guide learning experiences so that desired educa- 
tional results devel 3 objectives) can be achieved 
while the people are achieving their level 4 objec- 
tives. Failure to recognize the difference between 
fheso two levels of objectives has caused much 
ineffective teaching. It contributes to the forma- 
tion of detrimental attitudes and to reduced pai tici- 
pation by the very people who should be most con- 
corned. 

Stating Educational Objectives 

.Along with an undeislanding of levels of objec- 
tives, there is the p!*oblem of stating them in a form 
which will be most useful in developing and evalu- 
ating the educational pi-ogram designed to deal 
with them. This is an important pi-obieni for. un- 
less objectives are stated so that they ai'e useful, 
they may be filed away unused, and the educational 
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projiinni will not reflect the effort that went into 
their design. Well stated objectives, have two para- 
mount uses. The first is to facilitaie the selection 
and orj»anization of learning expei icnccs; the »sec- 
ond is to give explicit d.irection to the evaluation of 
educational program efforts. 

One. rather common practice is to state objectives 
;is activities or t lungs to bQ done which the exten- 
sion agents, or <5thers responsible for the program, 
plan to do for people. EXc^mples of this might be 
to show how to refiJiish a piece of furniture, to 
show how to use insulation materials properly, or 
to t^esent the agricultural adjustment problem. 

Such statements indicate what the person pre- 
senting the program plans to do, but they do not 
say who is to be involved or what is to be accom- 
plished by those involved. Statements in tei ms of 
activities to be carried on fail to indicate the kind 
of change oi* accomplishment . that is expected. 
Activities are not objectives; they are merely the 
means designed to accomplish objectives. 

Sometimes objectives are ijtated in the form of 
general content ai'eas to be dealt with in an educa- 
tional program. Statements of this kind do not 
specify what is expected of the people involved. 
For a group concerne<rwith health, the objectives 
may be stated 6y listing headings such as: Sanita- 
tion, vaccination, health insurance, and. the like. 
Cnless they follow in subheadings, such a listing 
does not specify the desired changes tb be made 
by the people. 

If. as we stated in Chapter II. education and learn- 
ing are viewed as processes for changing behavior, 
then'^objeetives may be thought of as the desired 
changes. Behavioral changes always occur in rela- 
tion to some particular situation .aud in reference 
to some particular person or group. Statements of 
objectives which give clearest direction to the ex- 
tension worker are statements which specify the 
behavior desij cd. ih(^ content in which the behavior 
is believed to be appr jpriate. and the people or 
groups who ai'c expected to change. The following 
objectives at the bxt^Mision worker level include the 
three elements necessary in -statements of educa- 
tional objectives. 



People 
Concerned 



Kind of Behavior 
Change Desired 



Mcnibors of th(* 
homonuikrr clubs 
CoiiMime/s of rc- 
tnil food.s in the 
Atlnritic Tfcido 
/Vrea 

Farm an.'l honio 
developmenl 
fanii Hos 

Farmors in the 
I.itlle Sioux 
Watershed 



To increase 
nndiT.standin^* of 

To increase 
knowlcdj^e of 



Problem Area 



The proposed 
comnuinitv center 



To develop 
interest in and 
understandint* of 

To develop the 
abilj!jy to 



The sources of 
food marketin{4 
information 

H;)w to make 
a farm and 
home P'On 

Establish ~^ 
contour lines 



When objectives are stated so that these 'elements 
are present, it is relatively easy to determine what 
kind of learning pxperiences should be planned, 
and what the content of these experiences should 
be. This degree of explicitncss in stating objectives 
is also adequate to give direction to action for 
appraising and evaluating results. 

The kind5 of experiences people need to gain 
understanding or to develop skills are different 
from those required to develop mere 'awareness or 
interest. Therefore, determination and statement 
of objectives are important tasks that require the 
broadest possible insight into the processes of edu- 
cation, the psychology of learning, and the sociology 
of groups. 

Criteria/for Judging Extension Educational 
Objectives 

V'e stated earlier in this chapter that objectives 
are direction givers for action oriented educational 
programs, and for evaluation of such programs. 
If objectives are to. serve these two important pur- 
poses, it is desirable to establish standards which 
may be used to judge their usefulness. Here is a 
series of questions which an extension worker, a 
conmiittee. or any individual or group might use in 
judging educational objectives. Ideally, a review 
of objectives for an extension educational program 
should result in positive answers to all nineiof the 
following questions. * v 

1. Are the '^objectives explicit in specifying the 
area in which the changed behavior is to 
operate? 

2. Are the objectives definitive with respect to 
kind of behavioral cha-nge to be accomplished? 

3. Are the objectives stated in terms which iden- 
tify those who are to be involved? 

4. Are the objectives the result of cooperative 
" action by extension personnel and others cpn- 

cerned to analyze the situation and idejitify 
the problems? 

5. Are the objectives compatible with the gen- 
eral aims of both the Extension Service and 
the people cdncerned? 

6. Aro the objectives specific enough to serve as 
a base for planning, conducting and evaluating 
an action educational program? 

7. Are the objectives sufficiently limited in num- 
ber to avoid undue confusion and diffusion of 
effort on the part of extension personnel and/ 
or others involved? 

8. Are the objectives achievable, considering the 
level of concern, the maturity of persons in- 
volved, and the resources available? 
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9. Are the objectives such that they can relate 
intimately to both immediate and long-time 
educational goals, and lead to even higher 
levels of achievement? 

'I 

Steps in Setting Goals and Objectives 

Extension workers and those with whom they 
work often do not take time to define educational 
goals and objectives. Sometimes there is l esistance 
to taking time to define them. This is reflected in 
the thinking implied by such statements as, **We 
don't neec\ to waste time doing that/' or ^'Let's get 
on with the job." Another very common feeling 
about setting objectives is expressed this way, 
*'\Vell, everyone Knows what it is we are trying 
to do." 

One imjDortant .step to insure the setting of ade- 
quate educational objectives, and their ultimate 
acceptance, is to provide time and opportunity for 
the people w^ho are concerned to participate in their 
determination. Goals and objectives provide little 
motivation for people who do -not take part in their 
determination. 

^ Another very important stej) is to explore the 
ideas and suggestions of those w^ho identify them- 
selves with the particular need or problem. This 
provides an opportunity for interested people to 
e.xplore alternatives which might not have occurred 
to them as individuals, and to secure some sort of 
consensus. After this is accomplished, statements 
need to be sharpened up sbv that they set forth 
explicitly the three necessary '(|imensions of well- 
stated objectives, as outlined earlier in this chapter. 
These statements then can be tested against appro- 
priate criteria for judging objectives. Once goals 
and objectives are set, there should bo a final check 
to be sure they are acceptable to and understood 
similarly by all concerned. 

Objectives that have been determined and ac- 
cepted Jn this manner meet the criteria for worthy 
extension educational program objectives and will 
serve as a firm base for both measurement and 
interpretation types of evaluation. . 



Summary 

We start this section v\«dth a discussion of objec- 
tives and how they fit into the scheme of extension 
teaching and evaluation. By taking the time to 
state theni, w^e crystallize our thinking on the prob- 
lem at hand and have a better idea of how to pro- 
ceed. In so doing we assemble oiu* forces in such 
a manner that all elements are directed toward a 
common end. 

We mentioned that there are group and individ- 
ual objectives, that they can be broad or specific, 
and that they can be immediate or ultimate. \ye 
also classified them for use in Extension as objec: 
tives of society in general, of the specific organiza- 
tion or group, of the extensioii agent, or of the 
people (clientele). 

Next we discussed how objectives can be stated 
so that they are limited in effectiveness by neglect- 
ing one or more of the requirements of a well stated 
objective: Who is Concerned?. What kind of be- 
havior change is desired? What is the groblem 
area? We also included nine. test questions to help 
determine how well stated an objective is in terms 
of extension educational goals. - ' 



References 

1. Association for Supervision and Curriculum Devclo'p- 
* ment, Guidance in the Curriculum; National Education 
Association, 1955 Yearbook, Washington, D. C. 

' 2. Brown. I. S. ''Working Toward Goals," Adult Leader- 
ship, Vol. 4, No. 4, p. 4-15. September 1952. 

3. Rurton, V^'. H. The Guidance of Learning Activities* 

New York, D. Appleton-Century Co., 1944. , 

4. Joint Committee Report on Extension Programs, Poli- 
cies, and Goals. USDA and Association of Land-Grant 
Coilegcs and Universities, Washington, D.'C^ 1948. 

5. Lancelot, W. H. Perm«ine(nt Learning. Ndw York, John 
Wiley and Sons, Inc., 1944. 

6. Tylcr, R. \V. Basic Principles of Curriculum and In- 
struction. Chicago, University of Chicago Press, 1950. 



'1? 



[21]. 



Chapter VI 

THB PLACE OF TEACHING PLAN 
ANALYSIS IN THE EVALUATION 
PROCESS 

Mrs. Laurel K. Salh orsk}/ 



Wc cannot find the worth of (evaluate) any 
activity or program, or estimate the probable worth 
of a new activity or program, unless we know the 
'educational and organizational objectives of the 
program. It is just as clearly a fact that we cannot 
evaluate adequately without a clear picture of the 
program that was followed in helping thq people 
to reach those objectives, or of a contemplated pro- 
gram that will be followed. This is a basic and 
vital step in the evaluation process, 

To extension workers; evaluation is determining 
how effective their work is, or how suitable their 
plans of work are for doing effective extension 
work. Therefore, we in Extension must find out 
if our work has contributed, or will contribute, to 
the accomplisHment of desired results. 

' This is the stage in the evaluation process w'here 
we study the methods used, or to be used; the sub- 
ject matter taught, or to be taught; and the audience, 
reached, or to be reached. This kind of analysis 
does not answer the qjTestion, '*Was I effective?** - 
Insteadr it helps answer the question, '*Have I 
arranged for a plan that has promise of resulting 
in an effective program?'' 

At this stage, we determine w^hat the objectives 
of the teaching plan were really aimed at during the 
.^Ume the project was being carried. Because exten- 
sion evaluation is finding the w^orth of extension 
work, or assessing the potential of. contemplated 
extension work or some part of it, we cannot evalu- 
ate in terms of objectives w^hich we, as individual 
extension workers or as part of an' extension team, 
made or will make no attempt to reach. To take an 
extreme case, how satisfied wolild you be 'with 
evaluation in terms of objectives if not one activity 
that you carried out could possibly have led the 
people in the direction you had wished them to go? 

We should also realize that analyzing the ppssi- 
bility of reaching an objective through a study of 
the teaching procedures and subject matter will 
provide a partial answer to the ultimate effective- 
ness of our work. By carrying out this preliminary 
step in evaluation, either when planning a teaching 
procedure or when planning an evaluation project, 

' . ' [22 



we determine whether the objectives were realistic 
or idealized, material or abstract. 

If we find that no attempt was made to reach the 
objectives, we can anticipate that the work will be 
ineffective in terms of those objectives. If w^e 

, conclude that inadequate methods or insufficient* 
subject matter were used, w^e can expect the 
effectiveness will 6e impaired proportionately. If 
we feel that objectives were, properly stated, appro- 

. priate actions were outlined, and adequate subject 
matter was provided, we can anticipate gratifying 
results in the way of desired behavior changes. 

Learning Experiences 

In educational extension work, the next step after 
clarifying objectives is planning for and defining 
the learning experiences to be provided for the 
learners. Essentially, learning takers place through 
the experiences the learner has and the reactions he 
makes to his environment. It does not take place 
through what you do as a teacher, but through what 
the student goes through as a learner. There may 
be only one teacher in the teaching situation but 
there are as many learning experiences as there are 
learners participating. v 

In the evaluation process, we assume that^^plan- 
ning for the learning experiences has been carried 
out, and proceed to analyze the plan. In order to 
make both planning and analysis more meaningful, 
we have some principles for selecting! and judging 
teaching activities (program execution). If plan- 
ning for the learning experiences does not take into 
account the following principles, its effectiveness 
might well be questioned without further examina- 
tion. 

1. The learning experience should be appropriate 
to the level of educatio)i, interests, and facili- 
ties of the learners.^^ 

2. The learning experience should be the con- 
sidered choice from among the possible ways 

' of accomplishing the desired behavior change. 

3. The learning exj^rience should be set up so 
that the learners have a chance to practice the 
desired kind of behavior. 
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4. The learning experience should be such that 
the learners will get satisfaction from carry- 
ing out the desired kind of behavior. 

5. The learning experience should lead, to in- 
terest in accomplishing other desirable be- 
havior changes. 

Let us take an example from extension work. An 
objective in your 4-H Club program may be to help 
boys and girls in 4-H Clubs to acquu^e the ability 
to perfonn the duties of club officers. Such an 

" objective has such long standing approval in exten- 
sion circles that it requires no evaluatton in its own 
right. A more appropriate evaluatron for your own 
purposes might be to determine tokvhat degree the 
club officers are functioning as they should. 

You believe -that the club president should know 
parliamentary rules and procedures, and be able to 
enforce them diplomatically. You believe that he 
should be poised and confident, and that he should 
speak^^ clearly and audibly. You believe that the 
secretary should keep complete and accurate rec- 
ords of meetings and be able to read them clearly 
and audibly. You have similar criteria for the 
other officers. 

Before you can begin looking for evidenco^that 
the officers are functioning properly, you need to 
do some desk work. Put dow^n and. study the activi- 
ties you/ as an extension worker or as part of the 
extension team, carried out. This should include 
the learning experiences you provided for training 
local leaders to train the 4-H Club officers. 

Did you arrange for the 4-H members to be taught 
how to do each of the-things that you are expecting 
of them? Did you see to it that the information 
needed to do these things reached every person 
who needed it?, Was it in terms that *^ey could 
understand and use? What knowlec" skills, or 
attitudes ought tor change in these boys and girls 
as a result of the teaching activity that took place? 
By such thought patterns, we move from '*What 
behavior change does the objective imply?'* to 

^ **What behavior changes could the experience 
bring about?'' 

• How did you do your work?" What did you try 
to teach? Whom did you try to teach? What did 
you expect the learners to know (to be able to do) 
as a result? Questions such as these fit into the 
pattern of the preferred way of stating educational 
^ objectives so that they include the three compo- 
nents outlined in Chapter V. 

Analysis of the Teaching Plan 

After you have clarified exactly what your ob- 
jectives are, what goal you hope the learners -will 
reach, what they can be expected to learr\ or acquire 
as a result of the teaching or training, and wKich 
ones can acquire the knowledge, you have the back- 



ground for an evaluation project on your teaching 
plan. After thesp are made clear, the other steps 
in the evaluation process are much easier. 

Again we want to stress the fact that evaluation 
includes not only, a measurement of the outcome 
of your work, but also an analysis of activities con- 
tributing to the outcome. This analysis' ought to 
take place during program planning, or during 
execution,' so that urgently needed modifications 
can still be affected. If it was not done at either 
of these stages, it must be done as part of the 
terminal evaluation to. eliminate the possibility of 
undesirable actions being repeated. Because of 
the importance of evaluation as a learning experi- 
ence for extension workers, it is essential that only 
thos6 directly involved in the woi'k do the evalua- 
tion. ;. ^ • 

We have found that a systematic method of carry- 
ing out this analysis, following -a definite outline, 
results in the most satisfaction to the evaluator.- 
The following form has worked out very well. The 
components suggested by the headings are essen- 
tial; the exact form can be adapted to the wishes 
and needs of the evaluatar. 



Teaching- 
level 
objectives 


Subject- 
matter 


Learning 
experiences 
(teaching 
methods) 


People 
to be 
reached 


Behavior 
changes 
expected 
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The order of these headings follows the sequence 
appropriate to planning a teaching program, or for 
analyzing one already set up. First is the matter 
of developing and stating the teaching-level objec- 

, tives. The *teaching-level objectives referred to 
here are the extension agent*s objectives for the 

.given program or unit, as described in the previous 
chapter. 

Second, for each such objective, certain subject- 
matter resources are required. List each kind of 
appropriate subject matter that was used, or that 
you plan to use,jafter^each objective. 

Third, you list those learning experiences which 
you feel will enable the learner to acquire the 
subject-matter understanding he will need to make 
the desired behavior change. Although it may be 
possible for you to list many different learning 
experiences after each type of subject matter, list 
only those that you used, or plan to use. However, 
you can keep the others in mind as alternatives in 
the event your evaluation indicates that your, 
choice was not the best one possible under existing 
circumstances, and there is time to make adjust- 
ments. 

Fourth, you list the kind of people who can be 
expected to receive the subject-matter information 
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. through each learning pxpericnce listed. For ex- 
ample, •'those who atleiid tlio ineeting" are the 
ones to be listed if some kind of a meeting is tlie 
leaVning experience. ''All farmers in inaine of) 
arer^ with an interest^ in dairy'* might bo listed if 
the learning experience is reading a news article 
or listening to a radio bioadcast devoted to a topic 
like the "wcigh-a-daV'a'month'V plan for improving 
milk production. 

F'ifth. iho expected change of behavior is listed. 
At this stage, the most important evakiation of the 
'learning experience takes phice. The kincl of learn- 
ing experience limits the kind of changes of be- 
. havior that can be expected to take place. Some 
learning experiences can be expected to change 
attitudes: others can be expected to change skills. 
Some can bring about both kinds of behavior 
change: others may have difficulty accomplishing 
one. ' - ^ ' [ 

For. example, it would take a homemaker with a 
very broad knowledge of sewing to learn the skill 
of making slip covers by listening to a speech on 
the subject. ^Jo!•e than likely, she would be the 
sorl. af person who already possessed the skill but 
just needed to have her memory refreshed on a few' 
points. Most of us would ne-iid the added experi- 
ence of watching a demonstralion. backed up by an 
opportunity to make one under supervision. 

When the above analysis has been completed, 
you have a basis upon which to decide which learn- 
ing objectives you are justified in studying, and 
which changes in behavior you can expect }o bring 
about. This systematic method of analysis can be 
applied with equally beneficial results to a fai'm 
-visit, a tplephone call or a club meeting. It can 
be used as an mtci:mediate evaluation of anything 
from a routine project to your total extension pro- 
gram. 

Relationship Between Judging Learning 
Experiences and Educational Evaluation 

, Judging learning experiences cannot be carried 
out without also judging the subject matter used 
and the kinds of people you expect to reach, as well 
as 'relating , the learning experiences to the objec- 
tives in mind. When this broad conception of 



judging learning experiences 'is emplo%'ed,_ it be- 
' con]es a large part of the'total evaluation procedure. 
The part that is left to do is to determine if the 
expected behavior changes occur, and how satis- 
factorily. 

Stated — Luagfly, judging learning expedenccs 
helps you decvidewlH-ch behavior changes to expect; 
cohiplcling the evaltatibji.proc'ess helps you 'deter- 
mine which expected behavior changes took place, 
* and how well; | 

Summary \ 

This and the succeeding chapter are devoted to 
intermediate types of evaluation of special u^e in 
cxtensioii teaching.^-^lfi this chapter we have jiis- 
» cussed how ev^jitf^tion enters into judging the prob- 
able effectiveness of a plan of work before there 
, are any results to use in the assessment. 

In evaluation of this variety, we are concerned 
with assuring a realistic approach to the problem 
at hand, by stating objectives in a meaningful man- 
ner, setting, up the action to be carried out, and . 
designing suitable learning experiences. Evalifa- 
lion is in terms of determining how appropriate the 
methodology is for accomplishing the objectives 
outlined. 

We have. presented a five-part outline forevaluat- 
ing an extension plan of work. It includes appraisal 
of the teach'ing-level objectives, the subject matter 
to be usdd; the learning experiences to be set up, 
the people to be reached, and the behavior cfianges 
to be looked for. At this stage we have to'compare 
the elements of the plan as outlined with approved 
extension practices. Later, when the plan has been 
executed, we can evaluate the program in terms , of 
observed behavior changes brought about by the 
action. 

-■^ 
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Chapter VII 

EVIDENCE OF PROGRESS TqWARD 
OBJECTIVES 

Mrs', LftHt'cI K. .sV//>/'o.s'/,*// 



You are well on your way with your evaluation 
plan after you have selected and defined the objec- 
tive for evakiation and the work you did to reach it. 
These lead you directly into determining the piiv-^ 
. pose of the specific evaluation you wisli to carry > 
out, with the necessary clarification and limiting 
of it. 

The next step is to determine \vhat evidence you 
will need to look for so that you will (1) know^ the 
situation, and (21 know that you are going in the 
right direction, or (3) know that your objective has 
been reached. This ia true regardless of the eval- 
uation procedure you will follow (use a record form, 
make a homp visit; carry 'out a formal survey, or 
others). ^ 

Evidence 

Our use of the word "evidence" is in ac^coyd with 
di'jtionary definitions: An outward sign, an indica- 
tion, that which furnishes any niodeofproof. " 
; Flow can you tell that you have/^rKuiFpi'ogress, or 
that you have attained the goal toward which your 
objective was aimed? Education is successful only 
to. the degree that it has caused changes in the 
"right*' direction. What is the evidence (the out- 
ward sign, indication, proof) of "right'* status 
change dr progress?, 

' Evidence. is not always easy to come by. Human 
beings are* complicated creatures and changes 
occurrinjj among them are not aJways obvious to 
the unaided eye. Thfs forces us to look for evidonce 
of the success of our objectives in terms of Observ- . 
able behavior, or res!.iIts of|behavior. (e.g. 'Learn- 
ing how to make a dress: Change in behavior; the 
dress itself: Result of behavior.) 

Behavior, when considered as a result of cduca- 
cation, is often not in terms of readily observable 
physical action. However, w^e can find evidence 
of it by lookin;: for improvement in skills, better 
umlerstanding of con<?epts, increased ability to 
soFve problems; changed attitudes, appi-ociation of 
different things, shifts in .values, changed interests, 
increased knowledge, adoption of. approved prac- 
tices. * - v. 

To select the types of behavior to- measui e, you 
must first jeheck tht' types of change you have been 



)r will be trying to help bring about. This involves 
reviewing y3ur plan of work. Different teaching 
methods or techniques have different types of 
behavior .Jiange as their objectives. The important 
thing is to ask yourself courtantly, Was this change 
supposed to occur because this particular objective 
is in the teaching plan? Were these methods the 
kind that could J)ring about successful accomplish- 
ment of this objective?" 

A radio talk may be aimed at producing a desired 
actid^j or at increr.sing the knowledge of the listen- 
ers, A recipe in a^newspaper may be put in for thc^ 
purpose of encouraging people to use food Jiiore 
economically,^ tastefully,, or nutritionally. A radio 
talk; new^s article, or project lesson may be'aimed^ 
at teaching ^ why something js important. One 
. method may attempt to bring about a, change in 
practice; another may attempt to bring about a 
change in knowledge, attitu^le. or appreciation. We 
shall need to decide v/hich and how many types, of 
behayior we want to measure. 

Depth of Evaluation 

1. In Terms of Changes in Behavior of People — 

'One level at which to measure your progress or 
attainment 3S a helper, educator, teacher br organ- 
izer is to find out which changes have tikeh place. 

a. In the People Themselves — Have th&y (the 
farmers, homemakers, young peoplel^changed their 
attitudes or skills; have they done anything as a>. 
result of the extension activity or method; has their 
status changed in any way ^since the extension 
action was staiHed? c 

b. In the Extension Workers (the* county 
agents, local leaders, c^- officers) being taught or 
-trained so that they' can help others l<>arn some- 
thing new/ 

2. In Tern^s of Opportunity — When it is difficult 
or iiHpo.ssiblc to Pleasure progress' at the level of 
oi'iginal status or change in people themselves, it is 
desirablo to measure work-in terms of. the learning, 
situation we have set up. If no learning situations 
are set up ino written materials go out; 7io talks 
are given, no demonstrations are put on, no visits 



aie made), we cannot expect the people to learn"^ 
anything as a result of extension work, v 

It is logical to assume thtit meetings which have 
1.000 people in attendance have a greater chance 
of changing behavior than no, meetings at all. How-^ 
ever, it would d(Jpend on the learning situation 
wheth9r a meetu^g^^th 1,000 in attendance, 10. 
meetings with 100 in attendance at each, or 100' 
meetings with 10 in attendance at each would' pro- 
vide the best learning opportunity. 

In general, the mtTre opportunities you provide, 
and the better they are. the better is your chance 
of accohiplishing your extension objective. A well 

* organized and well a*tt(?nded 4-11 Club event should 
offer a better opportunity for boys and girls to 
learn than a poorly organized and poorly attended 
one. ' ' ' . 
• By. apprai&ing the soundness of organization, and 
the attendance, coverage and subject matter, you 
have some inforlnati >n about possible success. You 

* still have no real qualitative measure of educational- 
change in behavior of the people who jire reached, 
but you have a good idea of quantity. 

Levels of Behavior 'Change at Which to Measure 

There are three convenient levels fordoing eval- 
uative moasureinents. The first ^one is before any 
changb, o^iiny further change, occurs {thd existing 
behavior/positioh before any extension teaching^has 
been .done, or before more teaching is undertaken). 
This peint. level, phase where the learners are be- 
fore the plan for changing behavior goes into action 
is referred to as the benchmark. 

The^decision whether or not to establish a bench- 
marJx has to be made at this step in the evaluation 
outline*: If you want to Icnow what progress has 
beefi made, or will be made, you should have some 
idea of the situation at the benchmark. 'If you know 
pretty well what it is. there is little need to estab- 
lish it formally. It is not necessary if you are 
interested mainly m whether the goal is reached, 
■ and not in how far learners had to progress to reach 
the goal. However, by establishing the benchmark, 
you provide yourself with a good starting point for 
program planning by finding out how urgent the 
need for change i-'eally is. 

The second level is during the progress of the 
learners toward the ultimate goal of the objective. 
Sometimjcs we think of evaluation at this level as a 
progress report. In evaluating a long-time objec- 
tive, progress reports are necessary to tell us how 
things are getting along. They serve to tell us if 
the teaching methods are effective, which ones are 
most effective and which gnes need to be empha- 
sized in the program of work if the objective is to 
be reached. They tell us which steps have, been 
accomplished and which remain to be accomplished. 
Also, they may suggest other steps which should be 



inserted because of- unexpected developments dur- 
ing the action. 

The third or attainment-of-goal level is at the 
close when, we must determine whether or not the 
final goal of the objective w. reached. When you 
find out if few, some, or all of the people reached 
the goal, you .know whether to retain the objective 
in the plan of wcJrk, redesign it and try again, or 
substitute another, one. You f:an find out whether 
teaching methods have been effective) or not^ and 
under what conditions. 

'Although it is desirable for most situations to- 
check at each level, two of them may be incorporat- 
ed into a single testing program. Because* it is 
necessary in most plans to establish a base for 
measurement, . the evaluation plan may call for 
measurement at two. levels, the benchmark level 
and either the progrc.ss level or the attainment-of- 
goal level. 

Factors for Deciding Which Evidences 
of .Behavior to Use 

Our next decisioi> concerns* which changes in 
people are to be considered as evidences of status, 
progress, of attainment. There are sevei'al factors 
whicli need to be considered in this regard. Most 
of them revolve around problems of measurement 
or ways to simplify measurement. ^ 

^ 1. Which Evidences Will^the Learners Have an 
Opportunity to Display? — Whether or not the learn- 
ers have the opportunity to develop a new kind of 
behavior, and to display it so that it can be detected, 
depends on two conditions. The first one is that 
the learning experiences must have occurred. We 
discussed the relationship of the learning experi- 
ence to behavior change in Chapter VI. The learn- 
ers must have had a chance to learn som.ething 
before it is reason^able to look for evidence that 
they did learn it. 

All of the people who had the opportunity, or 
those you wish to have the opportunity in the fu- 
ture, to Ifearn enough to make a change in* behavior, 
become the population from which you collect evi- 
dence. How to sample thi^ total .population, to 
make evaluation procedures manageable, is dis- 
cussed in Chapter X. 

The secoiid consideration Is whether or not the 
people had a chance to display the change of be- 
havior. This means that it is necessary to decide 
if the people have the facilities, the need, and the 
motivation to make the change of behavior a part 
of their thinking or- living habits. Only where a 
favorable or permissive climate for such changes 
exists isjrmeaningful to look for-evidences of teach- 
ing success. . 

This factor iimit^ the evidences much more in 
connection with youth work than it does with adult 
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education processes. It is true that favorable learn- 
ing experiences can-lje set up with ecjual facilily for 
youth educational gi'oups (the schools, VMW work, 
4-1 1 Clubs) or adult education work. However, a 
favorable climate for displaying evidence of be-* 
havior change is often, long delayed in t.'onuection 
with youth education work. 

/l. Wni the Things Yoy Can Count Be Adequate 
as Evidence? — These outward signs are the easiest 
results to us*e in evaluation, and they nuike much 
better working matei'ials for measuremc;nt tech- 
nicjucs. V However, if you do not consider tliei^G 
types of evidence as adequate proof of status, pro- 
gress, or goal attainment, you wiU have to think 
about Uays of assessing intangible evidences, such 
as changes in knowledge, altitude, or thinking. 

_ 3. What Is Your Own Facility in Measuring Dif- ' 
ferent Types of Behavior?— There is no question 
about the fact that it is easier to measure changes 
of'behavior which have tangible evidences for proof 
than -it is to measure those which havq intangible 
evidences. The fact that a homemaker has made 
use of a recipe she has seen in a paper is easier to 
determine than the tact tlfat shb now has a better 
understanding of the value of good • nutrition in 
child development. The fact that a family gathers 
around the kitchen table to plan the dirties of each 
member is not difficult to ascertain. But. whether 
or not the 'family now gets along better as a result 
^of this planning experience, is an intangible result 
that is hard to measure. 

To determine the real'^esults of extension teach- 
ing, we cannot ignore intangible results just be- - 
cause they are hard to ineasiu'e. On the other hand, 
while we are becoming accustomed to evaluation 
procedures, it i.<^ 'highly desirable that we become 
familiar with the techniques, metliods, ^nd pro- 
cedures by measuring those i-csults which i)rc easier 
to measure. When the techniques arc mastered, . 
we can use them to measure those types of rej?Ults 
which are more difficult to measure. 

Just as ive learn to read by starting with simple 
words, simple sentences, and simple stories, so we 
learn evaluation methods by starting with the 
simple and advancing to the more difficult measur- 
ing devices. Keep^in mind, however, when you 
begin with the easier methods, that you do not get 
the idea that all' evaluation is concerned only with 
tangible results that can be seen and .counted. 

4.^ How Will You timit the Number of Evidences 
to Look For? — If the list of evidences is very brief, 
and to look for all of them would not require a great 
deal of work, all of them should be included in yrur 
plan. If the objective to be studied has been de- 
fined and limited so that it requires only one or a 
very few actions or changes in behavior to reach it, 
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those a!-o the ones, and the only ones you need to 
look for. ' , ■ ♦ 

More often, howevei*. Ihcre are numerous evi- 
dences of change of behavioi*. When such is the 
case, there ai-e three frequenlly used ways of 
s/?iecting which ones to use in ycur evaluation. 

a/ Study the list of evidences to see if there 
are any readily apparent' typos of changes that, by 
, occurring, would serve as clues to the fact that 
other . changes must have been brought about. 
Sometimes only these discriminating evidences, the 
ones which can be taken as indicators of others, are 
enoo-igh to look for. In a progress report, however, 
be careful that those actions which are selected as • 
indicators of others are adequate to establish status 
of steps in progress. 

• b. If there are no siich discriminating evi- 
dences, or- if there are too many of them, study all 
observable changes to see which can serve as the 
most important indicators of change in behavior. 
If you cannot lookl'or all changes, certainly tliose 
you .consider mostjhipt rtapt will be of n).ost value 
to you. 

c. If your long list caiipbt be reduced by 
cither of these- methods, you-cfeii reduce it arbitrar- 
ily by -random selection'~--chbosing ct random a 
practical- number of them fo^ gareful study. Ran- 
dom methodology wjll be d1{&&sed in Chapter X. 

5. What Will You Do A^ppl the Time Factor? — 

Seldom is a change of behavior, whatever the type, 
apparent immediately upon recep^:ibn of tht teach- 
ing stimulus. For certam tvpes of change, ^uch as 
in attitudes? appreciation, Icnowledge, or y^alue§. a 
chajige in thought processes may take-place immed- 
iately. Ffowever, the learner will need time to 
think over the material, weigh it in his mind, and 
be exposed again to his customary sources of guid- 
ance, before a change any permanence takes - 
place. 

Knowledge which is evidenced immediately after 
presentation of new infomiation may be forgotten 
just, as quickly. When the desired change in be- 
havior is a change of practice, the time element is 
extremely important. Some things cannot be done 
until. ' .^or example, a certain season has arrived, 
certain materials are assembled, money, becomes , 
available to buy what is needed, or the faihily is alU;;:^ 
together. Some things, such as remodeling a house. ' 
take a long time to do. Some things take place in 
stages — planting tlAp%ar,den, harvesting it, canning 
the produce, serving ba'lanced meals throughout 
the year. 

In planning the evaluation, therefore, it must be 
decided how long it rs reasonable for you to delay 
your decision as to whether or rwi a certain practice - 
has been adopted, ^fidgment nuhyt be used in this 
pliase. We do not know exactly how long we should 



wait nftor a person has IkuI a chance to caiTV out a 
l^raclicc. and docs not do so. bcloro deciding that 
he is ntjt .^oing to do it. The^longei* he retrains, 
certainly, the less likely it is that he wilhever carry 
it out, ^ ' , ' 

On the other liancL there is a limit to the time 
we can wail and still expect to I'ind out exactly what 
was acconiplishv^d.- For daily happenings, 2 or 3 
days may bracket the nienioiT span. Regarding 
seasonal activities, we might bo justified in expect- 
ing a respondent to remember an important hap- 
pening duri^ig the rest of the year. 

When reporting mei.iiory items, it should be made 
•clear that the data are from memory and not from 
i:.ecor(ls. The pa.s.sage of time distorts accuracy of 

• recall considerably' in excess of a straight . line 
relationship. Periodic oh rvations spread over the 
peri(Kl. rather than a- sin.^le tormina! observation, 
will impro\'e accuracy of recall. For some types 

- of iiiformj^tion. you n^iy find it best to collect data 
at different times, because the various actions or 
, desired behaviors may be affected differently by 
^^t>Uie. 

(>. What Decisions Must You Make Regarding 
Units? — Will the evidence be by degrees of attain- 
menrr*rmiounts of things accojnplishcd. oi- length 
<A time an acpon has ibeen carried out? Will evi- 
dence of a year-around gai'len be in terms of 
certain ve^jjetables raised, amounts of certain vege- 
tables eanned and stored, or in other terms? Will 
it be the average number of (piarts of vegetables 
, per person, or will it be the percentage of the 
homemakers who' can a recommended average or 
niininunn amount of vegetables? 

What will be the evidenc'O you want or need to 
forn\ a basis for your analysis? What will be the 
units*.' Will they be conipai'able, or will you have 
to convert them before you can make comparisons? 

7. What Face Data Will You Need?— We think 
of those items of descriptive or background infor- 
mation needed from or about all individuals from 
whom we. get our evidence as face data. They in- 
clude su.^;h information as years of schooling, age, 
.sex. size of farm, number of children in the family, 
income, and so on. Face data arc used for three 
major purposes. 

a.' Classification of Respondents-^Usually it 

is desirable to find out if there is a differcmce in the- 
replies or I'eactions of individuals possessing differ- 

• ent degrees of certain characteristics. This requires 
an analysis of the replies using the classification 
system .set up for a particular characteristic as a 
sol-ting device. For example, if you want to know 
if there is a difference in replies among those who 
have had different amounts of formal schooling, 
YOU would need. to" include a face data question on 
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number of years- of formal schooling. If part-time 
farmers are to be compared with full-time farmers, 
appropriate face data tiuestiions' must be included 
to provide that information.! 

b. Purposes of Standardization— To compare 
the findings from one study with those from an- 
other, the individuals in both studies must be com- 
parable, at least in terms of the sort factors under 
consideration. The findings in a study of commer- 
cial farmei's are not directly comparable with find- 
ings in a study of subsistence farmers. If either 
or both studies TSiyi/no face data questions on size 
or type of farm, the findings from the two cannot 
be standardized .for puri)osos of studying the rela- 
tionship of size of farm o{>eration to other aspects 
under consideration. 

c. Purposes of Comparison — Frequently it is 
. desirable to compare data from a, study of 'only a 

part of the population with data from a study of/ 
the whole population (universe). Usually census 
data are used for such purposes because they deal 
with the whole population and because they are 
about the best source of depth or trend information. 
In such cases, face data questions comparable to 
those used to secure census data'should be included. 

For example, if the study data are to bo checked 
with census data on age, residence, or similar cla.ssi- 
iicalions. face data questions must be included 
which will provide data suitable for assembling into 
the categories used by the census. When compari- 
sons are made between studies or with cen.^ms data, 
it is vital that data be secured in sufficient detail 
so that they can be converted to comparable units 
or rcpovtmg intervals, although they do not have 
to be colle:*-ted in identical form. 

This means that in planning an evaluation, and 
especially while constructing questions, the analysis 
which will eventually be made of the results must 
be given serious thought. Think over the teaching 
methods and the informational materials. What 
do you want to know about the behavior changes 
you arc trying to bring about? What human re- 
actions -do you want to explore? 

Do you expect certain types of people to react 
.more favorably than others to your teaching? Do 
you expect that certain mental or material posses- 
. .sions or ciualifications will, make a difference? If 
.you do. be certain to include appropriate fafc data 
so that you can sort on these characteristics, and 
evaluate their effects properly, ^ 

Face data will help you ascertain differences, 
'make comparisons with other studies, and check 
• the probable accuracy of your data. All of this 
contributes to your major purpose in evaluation, 
truing your work aild making new ami better plans 
for your w^ork. 



.Summary ' 

•f III this chapter wc have probed inui'e intensively 
into the prublems ol evahiating (hiring the piogi'e.ss 
o| a program. During this clynairiic stage we must 
look for evidences or results which we can use to 
;issess the progi'ess of our work. We might prefer 
\" to look for them in terms of changes in pej[)le, but, 
\ because of the delay before most changes actually 
occur, we might have, to settle fur counting oppor- 
tunities provided for changes to occur. 

We suggested three stages when evaUuition is in 
order, referrfng to them as the benchmark, the 
progress report, and the attainmcnt-of-goal levels, 
and suggested some techniques appi-opriate tdeach. 
We then devoted the balance? of the chapter to dis- 



■ " ■ cas^;ing^sF\TTrllTrtin'5*'tD-rdi^^ 

evidences to use in evaluation.^ We gave special 
attention to the last factor, the matter of face data, 
because of iti? importance to the more detailed 
types of evaluative action v;e will be discussing in 
forthcoming chapters in this section. 
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Chapter VIII 

IDENTIFYING PROBLEM AREAS 
AND STUDY OBJECTIVES 

'Ward K Porter 



In this chapter we return from our exploration 
of the various types of evaluation to a brief account- 
ing of - the initial phase the evaluation process. 
This chapter is iiimed at directing our thinking to 
the "big picture'* of evaluation planning, before 
moving on to specific techniques of evaluation. 

Our immediate objective is to help the reader 
develop a better understanding of the process 
whe^reby appropriate research or evaluation prob- 
lems are identified and study objectives determin- 
ed. Wo hope to emphasize the urgency of doing a 
thorough job of understanding the problem befortj 
starting to evaluate it. All too frequently, the 
importance of this first step is not properly appre- 
ciated nor sufficiently understood. The, net result 
it that much needlessly slipshod evaluation is turn- 
ed out and, unfortunately, gets used without ques- 
tion in^policy making and program planning actions. 

The importance of Problem Identification 

Precise identification of the problem needing 
investigation is essential from many standpoints. 
This step helps the extension worker clarify his 
thinking and facilitates the. selection of suitable 
study objectives. It also serves as a basis for sound 
decision-niaking regaiiding evidence to look for, 
data to be collected, and analyses to be made. It 
.sharpens up his interpretations and suggests ways 
of getting the results into use. ^ 

The need ier evaluation and research in Exten- 
sion grows out of our role as educators and the 
resulting necessity for determinfil'^ the effective- 
ness of our programs and methods. Only in this 
way can we make necessary and realistic modifica- 
tions in future actions to assure maximum benefits , 
from our extension teaching. In addition, all ex- 
tension workers are confronted with the problem 
of identifying the needs and interests of their 
clientele. 

Identifying these needs becomes an objective. of 
sufficient importance to demand the most careful 
evaluative and research techniques. Although, 
some situations are such that relatively informal 
exploratory devices wiil serve, others may require 



full-blown,,formal studies. Whatever measures are 
used,' the results will be applied 'in determining 
action programs, in deciding upon appropriate 
methods, or in ertablishing benchmarks by which, 
educational progress can, " at - a later cjate, be'' 
measured. - . •/ * ' ^ 

Selecting a Problem for Study 

The fact that -you have identified a problem area 
docG not signify that you are ready to begin your 
^valuation. More often than not, the problem you 
have selected is much too all-inclusive to be handled^ 
adequately in a single evaluation. There is still the* 
matter of identifying the., specific pr^lem for your 
evaluation. 

There are many factors that need to be consider- 
ed befoi;e selecting our specific problem area and 
presenting it in a form suitable ior Qvaluation. We 
must consider our own personal and professional' 
needs and interests in'^the matter, as well as the 
needs and interests of our. associates, our lay lead- 
ers, and our clientele. Obyiously, there AviU b3 
some- overlapping of interests a-nd^co^ncerns, as \yell 
as occasional conflicts. Any differences, of course, 
must in. some way be resolved' ' ' ^ 

We must decide not only the general area of con- 
ceruj but also the particular phase or aspect that is 
to be studied. As ^ example, let us take the case, 
of the extension worker who would like to know 
how effective he has been in reaching a certain 
group of families with a particular pjogram. Al- 
ready he has to a degree, limited his area for study 
by specifying a broad problem area. >Before he 
■has a study he can cope with; however, he will have 
to narrow it down further. He might choose to 
select acceptance of a single practice as an indicator 
of how well he has succeeded with his educational 
program. In short, if his program- involved pasture, 
imprpvemej^f', he rhight select just one of the sev- 
eral recommended practice.s— periodic clipping, for 
example — for the- detailed study ■ 

'The d'ecisio'n as to which problem is to be investi- 
gated at any one time inevitably must require some 
judgment (evaluation) as to tHe relative importance 
of the many different problems. The^geht might 
try outlining worthwhile areas of study, some prob- 
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lems in each, ancrthe specific topics which could 
be studied under thoin. He nughl organize them 
into some form of priority Usting in terms of their 
probable importance^to himself, his associates, 
his; lay leaders or his cJientelc. Then, bv adopting 
thc^ policy of -first things first" .as in program 
plaiining^^-he can decide on his plan of action. 

Such a plan should be executed with care, how- 
ever, as failure to correctly assess the relative im- 
j)ortance of the many problems demanding investi- 
gation can result in frustration and discouragement. 
This is another way of saying that evaluation, like 
most things that are worthwhile, requires consider- 
vable planning. 

Time is another factor which must be given ade- 
quale attention. An alNtoo-frequent criticism of 
evaluation plans, and research in general is the 
inadequate amount of time allotted for the kind of 
investigation that is outlined. -^Yttequate attention 
nuL^l also be given to other types of resources which 
nuist he employed in the study, such as personnel, 
finances, skills and know-how, equipment and facili- 
ties, and related items. 

L:nfortunately. these resource factors are not 
always considered carefully enough in planning an 
evaluation.vor survey. Often the result of such 
oversight "is a study that never gets finished, some- 
times because funds arc expended before the data 
can be analyzed. Another distressinglv frequent 
ocOurrence is when reseai chi.-rs, with qn "impressive 
array of electronic equipment at» their disposal, 
dash into the field with a carelessly constructed! 
-shotgun" type questionnaire which is utterly in- 
capable of~s'ccUring valid, reliable, and objective 
data. 

One other important preliminary consideration 
is the amount and kind of information already at 
hand that may be relevant to the problem. Solu- 
tions, or at least objective information from which 
inferences can be drawn, may already be available, 
in this event, duplication would be a needless waste 
of time' and other resources. 

One note of caution is in order in this connec- 
tion. Inferences drawn from other studies or re- 
search findings of presumed relevance should not 
be applied to the new setting without thorough 
thecks for comparabihty. Overgeneralization of 
research-findings, without regard for comparabilitv 
of situations, people involved, research techniques 
used, and related factors, is altogether too common, 
and* sometimes fraught with serious consequences. 

Selecting and Defining Study Objectives 

Once the problem area has been identified, it 
becomes a matter of pinpointing the specific objec- 
tives of the forthcoming study, hi a sense, the 
^''"lUifjcation of the probleni(s) may be thought of 



as establishing the overall purpose of the investiga- 
tion. That is. a study im\y be undertaken to estab- 
lish benchmarks for later program or method 
evaluation, to indicate needs and interests of thq 
clientele, or to establish an estimate of the effective- 
ness of a meeting in changing 'hujnan behavior. On 
the other hand, the proper statement of studv 
objectives indicides specifically what aspects of the 
problem are to be emphasized. 

We can illustrate this distinction between iden- 
tifying the problem and setting up the objectives 
for study purposes by t!iis example. A home 
demonstration agent might have the problem of 
evaluating a leader-training meeting. Her study 
objectives might be as comprehensive as the educa- 
tional program involved at the meeting. At tho 
same tiuie, she might quite properly limit her 
evaluation, because of other-pressing demands on 
her time and other resources, to a single aspect of 
the program. She might, for example, be content 
with determining the number of leaders attending 
the meeting who learned how to freeze peaches, 
even though freezing procedures for several other 
foods were discussed. 

It is important in this connection to distinguish 
between educational objectives and study objec- 
tives. Previously, we defined educational objec- 
tives in terms of changes in human behavior (e.g., 
an increase in knowledge, understanding, or skills, 
or changes in attitudes or opinions). Study objec- 
tives may be phra^^l in terms of educational 
objectives, but do not have' to be, depending upon 
the nature of the problem. 

In a strictly sit.u{itional type of analysis, such as 
a countrywide survey, no teaching objectives as 
such need be directly involved. In an evaluation 
of the effectiveness of an educational effort, how- 
ever, study objectives most certainly would have 
to be consistent witii the program's specific educa- 
tional objectives. In either case, of course, the 
scope of the investigation may be limited. 

The study objectives also can be stated in terms 
of learning opportunities provided. Instead of 
attempting to appraise knowledge or skill,-? acquir- 
ed, the evaluaf i can be limited to determining the 
number of cxpo,^iires to the learning situation — 
how many attended the meeting, how^ many heard 
the radio program, how ina^iy made the field trip, 
and so on. As indicated earlier, such opportunity 
level evaluations have their shortcomings and 
limitations. 



Stating Study Objectives 

The factors mentioned previously as affecting 
problem identification, such as resources available, 
are all equally relevant in selecting, defining, and 
amplifying study objectives. Here again, it may 



• l)f hi'lpful lo list tiu' various polenli:il study objcc- 
^ liM'>. a j (Mii*nnipassecl by thc> i)rol)leJn area selected, 
iti ihe ordor (il Iheii' iiuporlanee and then consider 
ear!i in- Hie li.f^ht of resources available. 

\\v can help (Hir.selves a lot when it conies to 
(/oik'tiinji. analyzing; and inlerpi'cling otir data if 
v,e lit) a .^o(kI job ot statinj.; our objectives. The 
UKJie |)recisely ue state thejii the easier ll will be 
to desi^i;n i|ue.sti(Mi.s that u ill secure the infornuition 
v.e njrcd. i 

0(\v devieo forgaininfj precision in our statcniont 
of .sludy tibjcclives is to develop a series of basic 
(juo/<tioijs around which the investigation can be 
..iticused. In a study of the effectiveness of a mar- 
ket inji-inforniation-for-conVuiners (MIC) progranu 
for example, one of our study objectives might be: 
To iletermine the nature and scope of the MIC radio 
audience in **X" County. 

We could l)rin,^ this objective into sharper focus 
by listin.Lj several basic questions that we want 
answered Such a list might include questions of 
this o^der: ih Hovv many farm families in "X" 
( ounl\ have regular access to a radio that is in 
woiking ordei'V \2) How niany of these families 
<o!ie or more menibers ofi listen regularly lover 
haJ oi Hie broadcasts) to the MIC radio progi'am? 
\:h What prop;)i'tion of these families'live on farms? 
In the opeii country? In small towns or villages? 
In cities lover 1(),()(}() inhabitants!? 

In the more formal and comprehensive types of 
rivsearcli projects, researchers follow up their state- 
rneiU (d' objectives by working up appropriate hypo- 
theses relating to them, which they proceectto test 
by statistical and other methods. These hypotheses 
are their considered opinions, based on inferences 
of what they expect their research will find out 
; regarding their objectives. The hypotheses arc 
then tested by the data collected and either sup- 
poi'ted or rejected, in part or in whole. 

The hypothesis becomes the focus of the investi- 
gation, directing the attention of the researcher 
t{)ward certain types of facts. By so limiting or 
delineating the study, the random and unsystematic 
search for facts is avoided. 
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An example of one hypothesis that might be 
tested in an extension study is the followihg'4^iesult 
demonstrations are niore effective than the mass 
media in promoting the adoption of certain recom- 
mended practices among low-income faim families 
of '"X" County. To confirm or deny this hypothesis, 
u e would have to work up a series of questions to 
help us determine, (1) the degree of effectiveness 
of the result demonstration, and (2) the effective- 
Jiess of the mass media, both separately and collec- 
tively. First, however, we would have to select ^ 
suitable practices and devise some criterion of 
effectiveness— an exposure index, an acceptance 
score, or similar device. 

Only when the study objectives are precisely 
stated are we in a position to plan and carry out 
the other phases of our evaluation or study in an 
effective manner. Time and effort devoted to this 
initial phase of the process will prove very reward- 
ing in terms of the quality, reliability and usofu!- 
jiess of our results. 

Summary 

In this final chapter before elaborating on the 
techniques for assembling and analyzing evaluative 
data, we have briefly discussed ways and means of 
selecti.ng and stating our evaluation problem and 
study objectives. In essence this is largely a mat- 
ter of identifying problem areas of major interest 
and concern, selecting one or more for immediate 
investigation, determining our study objectives, and 
pinpointing the major questions or hypotheses that 
will serve as the focus of the study. Although 
simple-sounding in outline, each step merits careful 
and detailed consideration. The success of the 
entire evaluation process will depend on how wet! 
each step is worked out. 
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Chapter IX 
SOURCES OF DATA 



As ue staled in Chapter VIL it is very euiiinion 
for the evaluation process to start oil' with a bench- 
mark phase. The Heed to know where you are so 
you can d^jiterniine later how far you got is funda- 
mental to evaluation theory. To set up this bench- 
mark, we must learn what we can about the existing 
situation. 

If we are old timers on the scene, we can use our 
own experiences as a source for much of our setup 
data. It might also be possible to call on others to 
supply much of the information we need on the 
basis of their first hand knowledge of the situation. 
However, even. the most observant person needs 
information of a factual or statistical nature to 
supplement hisj)rst hand actjuaintance with the 
setting. 

For most of our purposes we will find, upon in- 
vestigation, that there is a great amount of niaierial 
already assembled which will be of use to us, for 
purposes of control and comparison if not for'actu- 
ally solving our problem. By using these sources 
wherever 1 whenever they can help with our 
evaluation \^e arc saving valuable time and other 
resources for use in later stages of our evaluation 
or for further evaluative actions. "^.^ 

Primary or Secondary Source 

Traditionally, sources of data have been classified 
into two categories, primary and secondary. Our 
major interest in this chapter is with the latter kind; 
assembly, of data from primary sources will be the 
central theme of the remaining chapters of this 
section. 

We can think of prirnary sources as the original 
documents, the first reporting of the facts, the first 
grouping of the raw data. Secondary sources bring 
together facts from primary sources. They can 
make a substantial contribution to knowledge by 
way of analysis and interpretation of primary 
sources, and by making information presented in 
obscure primary sources more readily available 
and understandable, but they do not make the orig- 
inal contact with the unknown. 

DaLi assembled by means of questionnaires, 
schedules, interviews and similar instruments are 
data from primary sources, Records from 4-H 



members, I'esult demonstrators, farm and home 
development cooperators, and periodic reports of 
extension personnel are primary sources of data. 
Summary reports of State and Federal extension 
officials occupy a somewhat uncertain position in 
this classification. In one sense ihey are original 
tabulations and compilations; in another they are 
compilations and inten)retations of data from 
original sources. 

Often we will find it difficult to make the decision 
as to whether a source can bo called a primary or 
a secondary one. For instance, reports compiled 
from census interview forms are primary sources. 
Reports compiled from census reports, irrespective 
of whether they were done by census personnel or 
people in other organization^i^ and agencies, must 
be classified, as secondary sources. Many of the 
reports we in E,\tension make or help to assemble 
contain both original and secondary^data. We will 
soon discover, however, that we can make good use 
of either kind by making proper adjustments and 
allowances, 

Although this decision is purely academic in most 
situations, it must be taken into consideration when 
evaluation is being planned, It will guide us in 
decldi;ig how much trust to place in the data 
through our ideas of the presumed professional 
stature of the author. For instance, most of us 
accept original research prepared by State and 
national governmental agencies and established 
research organizations with respect, if not without 
question or qualification. At the same time, we 
would want to make a personal decision regarding, 
the qualifications of a committee of dub members 
or a masters-level graduate .student before accept- 
ing their Holdings without question. 

By the same token, we would want our secondary 
solirce to be at least as qualified as the primary 
source it quotes. Much good primary research gets 
abused during the process of inefficient secondary 
analysis. Similarly, more than one presuriKibly 
sound research organization has committed the 
error of attempting to salvage inadequate original 
research by applying elaborate and unjustified 
statistical manipulations while synthesizing results 
of tfte original. study. 
There are reasons for preferring a primary 
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source over a ijecoiiclarv one when both arc avail- 
able. In the ease ol' tiie tbriiier. there are fdwer 
opj)ortunitie.s for erroi-s to get into the final manu- 
script. Counts, sorts, tabulati^ns and interpreta- 
tions are made by those most lively to understand 
the material. When special definitions are neces- 
sary, there is^ greater likehhood that tjjey will be 
u.sed consistently throughout the* manuscript. 
Groupings will be made in terms of the original 
material, rather than the needs of the secondary 
researcher who may never have an opportunity to 
examine the original data. 

Secondary sources may contain mistakes of both 
transcription and interpretation resulting from the 
extra handling :yid lack of fannliarity with the orig- 
inal dat;}. and the metliods by which they were 
assembl(^d. Quit'e often tiiere is a tendency to 
generali;>:e or extrapolate igo beyond the datal with- 
out adequate^ justification. This is not to suggest 
that secondaiT sources of data have no value, mere- 
ly that they may have limitations less likely to be 
encountered in the original source. 

Secondary sources may point out important pri- 
mary sources which we might ne\er learn of if left 
* lo our own I'esources. They are helpful in opening 
up an area and in suggesting directions for individ- 
ual exploration They show us where we can locate 
original sourcx's and what we are likely to find. 
Their interpretations may help us to understand 
the data of the oVi^inal report. If properly used, 
they can save us a great amount of time and effort 
without leading us astray from our objectives. 

Sometimes data from either source are suffi- 
ciently comparable so that they can be used to 
answ;er our questions, and further research on our 
part will not be necessary. If either is used in 
place of personal research we must be certain to 
check on reliability, accuracy and applicability as 
thoroughly as we can. This is an occasion when 
we can draw on our own family of experts — the 
research and subject-matter people at tbe land- 
grant instituHons and in the Federal Extension 
Service offices — to help us (ietide on quality of 
sources. 

Useful Data Sources for the Extension Worker 

Xo extension worker need fear that he will have 
to "go it alone" on a research problem. More than 
likely, he will be overwhelmed by the wealth of 
background n^aterial that can he made' available to 
him. once he gets tuned in on the proper communi- 
cation channels. 

It is difficult to conceive of a problem that will 
lie entirely outside the area existing research. 
Seldom will ynu find a piece of research which 
exactly fits your problem, but you can be assured 
that you will find .several close enough to give you 



I a lot of help in laying out your own study. If you 
\jieOd help with pulling these leads together, there 
, are persons qualified to advise you in the State 

Extension office, possibly even in the county, and 

surely in4he Federal office. 

Sources of both published and unpublished mate- 
rial are too extiMisive to be given comprehensive 
treatment here.s However, we will suggest some 
favorite starting points for the exploratory phase of 
an evaluation assignment. \ 

General— In our Work we are likely to think first 
of the U. S. Department of Agriculture. The vast 
research resources of the Department are available 
to us at little or nacost to us personally. Our con- 
tact for published materials of the Department is 
its Office of Information or the informaiion outlet 
for the apprcpriate\igency within the D^artment. 

We should become familiar with its Bibliography 
of Agriculture, published monthly by the depart- 
mental library. This contains listings of many, of 
the current agricultural publications, both foreign 
and domestic. Incidentally, this is a good institu- 
tion to become acqtriunted with ijf you arc in a 
position to come to Washii^gton, or can make use 

of its loan facilities. 
fx 

Another good starting point is the monthly pub- 
lication of the U. S. Government P-rinting Office, 
Monthly List of Government Publications. It has 
extensive listings of agricultural items^and indicates 
where free copies might be available, or else quotes 
prices if they must be purchased. In addition, 
several of the agencies have periodic listings/ of 
materials prepared within the agency. Often iris 
possible to be placell on their mailing lists to 
receive such notices regularly. Very often, free 
copies are ^jivailable for distribution, at least for a 
short while after publication,, from their informa-, 
tion branch or division. 

The Department or its agencies assembles a greiit 
many periodic and special purpose reports provid- 
ing a vast amount of ttend. estimating, production, 
piedictive and regulative information on agricul- 
tural subjects. It maintains xesearch experts, con- 
sultants and. technicians^ in the Washington offices 
and dispursed throughout the SU>te£ and Terri- 
tories, and has many others on- foreign asS^ment. 

The resources of the U. S. Department of Com- 
merce are also available lo us. We can make a 
great deal of use of the reports of its Bureau of the 
Census. To learn what this agency has to offer, 
we should know of its Quarterly List of Census Pub- 
lications. It lists virtually everything issued by 
the bureau and tells you where things are avail- 
able, together with the price if there is a charge. 
The U. S. Department of Labor also does research ' 
of interest to us. We can get valuable infonr,ation 
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on the agricultural labor force, the migratory labor 
force and like information from them. 

At the State level, we have the numerous facili- 
ties ,of the land-grant institutions (the^ Extension 
Service, the Experiment Station, the subject-matter 
departments, the agricultural college and experi- 
ment station libraries) at our disposal. Their pub- 
lished works generally are available without cost 
to residents of the State from their Agricultural 
Albiling lioom or some central office similaay titled. 

Many of the regional and district people are only 
too happy to help us. Much service can also be 
rendered by the State Departments of Agriculture. 
Then, too, there are the larger farm organizations, 
many of whom have their research and public infor- 
mation functions well established at both State and 
national levels. y 

Locally we have the facilities of the County Ex- 
tension Office, where n^any of the published mate- 
rials originating from the above mentioned sources 
are already available. We can call on the local 
civic organizations and local chapters of religious 
and professional organizations. Then, too, there 
are the other civic officials around the Court House 
and elsewhere'in the towns and cities of the county, 
as well as the elementary and secondary educators 
and education officials of the county. 

Home Economics — Most of the general sources 
already mentioned will give us leads to materials 
with special application to home economics evalua- 
tion tasks. However, we would also mention sev- 
eral agencies, or V^^iphes and divisions within 
agencies of the U. S. Department of Agriculture, 
whose task is developing knowledge about home 
economics. 

Research in phases of home economics has been 
carried on by a succession of agencies within the 
Department since 1894. In the present depart- 
mental setup, the work is being carried forward by 
the Institute of Home Economics in the Agricultural 
Research Service. Special phases of the work are 
assigned to the Clothing and Housing, the House- 
hold Economics, and the Human Nutrition Research 
Divisions within the Institute. 

Home economists also may be found in other 
agencies whe/6, although they may carry any one 
of several social scientist labels, will be working 
with the problems of the homcmakcr. Our own 
division employs home economists in its Federal 
offices and the agency maintains a separate divi- 
sion, the Home Economics PrograiTip Division, to 
advise on and conduct home economics research, 
among other things. Published works of all these 
persons will b.e found listed in the sources mention- 
ed previously. ' 

Other places where we should cast an interested 
glance from time to time, because of the valuable 



research they do in related areasf are: (1) The U. S. . 
Department of Health, 'Education and' Welfare, 
where they do so much work related to the home 
(rural health, child welfare, pure food and drug 

• work, rural education); [1] The U. S, Department 
of Labor, for its studies of women in industry (and 

* out of the home), and child labor problems; (3) The 
U. S, Department of Commerce and its Census 
Bureau with its many Agriculture, Housing and 
Population reports. 

As they do at the national level,' most of the 
organizations mentioned earlier have Home eco.no- 
mists doing home economics research at the State 
level. The land-grant institutions have schools' or 
departments of home economics where they train 
home economists and do home economics research. 

Most of the counties have home economists on 
the county extension staff to head up the home 
demonstration work and to participate in the farm 
and home development work. Most of these people 
report on research they have done from time to 
time, and the published reports became available, 
at least locally, for use of their fellow workers and 
their clientele. 

Extension — We have already mentioned some 
sources within Extension but wp would like to 
suggest a few more. One source, actually several 
sources, of new information on research are the 
newsletters of the various extension divisions and 
branches to their counterparts at other levels within 
the agency. Most of us at least have access to them, 
if we do not receive them directly. 

One of the special assignments of the Division of 
Extension Research and Training is passing along 
knowledge of research findings. To this end they 
put out a series of periodic reports of use as mate- 
rial or idea sources. 

They prepare annual digests of statistics compil- 
ed from the annual reports of county and State level 
extension workers. Each year they publish Review 
of Extension Research, which is a compilation of 
summary reports of research related to organiza- 
tion, functioning and administration of the work 
which were complete(^uring the previous year. 

At five-year intervals they publish Bibliography 
on Extension Research, which is a listing of exten- 
sion research reports published or reported on dur- 
ing the previous five-year period. To supplement 
these two reports they prepare special listings from 
time to time as conditions warrant. Recent topics 
for speeial listings have been radio, television, and 
communications research. 

Recently, the Division has begun preparing a 
series of popular presentations of results of exten- 
sion research. This numbered series is called, 
ER&T Research Summaries^ Distribution is limit- 
ed; presently the Federal staff and the State leaders 



of extension studies are receiving all summaries. 
However, interested persons can get separate sum- 
maries upon request. 

Local workers will find good source materials in 
their office files of annual reports and in their 
records of farm and home development work and 
rural development w^ork. Similar use can be made 
of 4-H record books and result demonstration 
records.,;^ 

Summary 

If we are beginning researchers (evaluators) we 
need not be overly concerned about background-' 
data for our problem. We just have to know how 
to get started at digging it out and then direct our 
energies to being selective of what we find so that 
we use only that which is proper and relevant to 
our work. 

In this chapter we have suggested that sources 

y 



of data ari j of two varieties, primary and secondary, 
and have stated a qualified preference for the for- 
mer. We also stated that many research, reports 
contaiaboth kinds of data and that often the classi- 
fication is difficult to make. We went on to say 
that, rather than make the distinction for classifica- 
tion reasons only, we should make it for purposes 
of determining how freely we should use the mate- 
rial in our own problem's. 

We mentioned some favorite ways of getting 
started on the task of locating background material 
for evaluation work and suggested sources at differ- 
ent levels wher.e research results are made known, 
or where we can direct inquiries about specific 
topics. We gave some leads for securing bench- 
mark and other data for general agricultural exten- 
sion problems then suggested some for home 
economics problems. We closed by mentioning 
some specific materials which are available from 
the Division of Extension Research and Training. 
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Chapter X 
SAMPLING 



Mrs. Laurel K. Sa^^rosku 



Extension workers usually reach and teach more 
people than they can hope to get evidence frpm to 
determine whether their clientiele has need for, 
ol- has experienced, an educational change. In a 
way this is unfortunate, as the way to get complete 
and reliable information is to go to all the people 
and find out from them, in one way or another, 
whether they need or have learned the things you 
are trying to teach. Since this is seldom possible 
in extension work, we have to get information from 
a part of the pqople about some of the things taught. 

When you choose only part of the people froni 
whom to get information, you are sampling. Since 
the methods v/e use in selecting the sample affect 
so greatly the reliability of the information we 
assemble, we need to have a fairly detailed under- 
standing of how sampling is done. 

Even though very fev^ of you may ever have 
occasion to develop a comprehensive sampling 
plaij, you do use sampHng methods more often than 
you may realize and regardless of how simple your * 
data collection effort may be. 

This means that sound principles of sampling are 
important to all of us in Extension. Proper sampl- 
><ng I principles and procedures should be in the 
" mind, pf: every extensipn worker whenever he is 
planning or evaluating his work. In the balance of 
this chapter we will discuss: some approved sampU 
ing procedures which have found a. place in exten- 
sion work. ^ 



General Definitions and Principles 

Sampling is sefecting a srnall number of units 
from among the whole group concerned in such a 
manner that they can be used to make estimates 
about the whole group. Too often, a sample is 
thought ^ of as consisting of only the first part of 
this definition: "... selecting a small number of 
units from among the whole group concerned . . . 
However, to be meaningful in statistics, and in any 
evaluative effort, the sample must be drawn in 
such a manner that the entire definition applies. 
It is essential that . . . in such a manner that they 
can be used to make estimates about the whole 
group" enters into plans for drawing the sample. 



Four steps are involved in sampling, regardless 
of how elaborate the actual execution may become. 
They consist of: . . 

h Defining the population. 

2. Choosing.and executing the sampling plan. 

3. Dravying conclusions based on * information 
extracted from the sample. 

4. Inferring conclusions back to the total popu- 
lation. 

Sample 





t. DEFIN'E 
POPULATION 



4. INFER BACK ^ 



■CONCLUSIONS 



The accompanying diagram illustrates ^ this 
sequence of events. The first two steps must be 
•involved in the planning stage; the other two con- 
stitute the analysis and interpretation stage. The 
soundness of the sampling plan will determine how 
faithfully we can generalize from our sample badk 
to the population. 

. The population for our purpose is the total unifs 
to be involved in oiir evaluation. In extension 
teaching the units are generally people (homemak- 
ers, club members, project participants, potential 
listeners or viewers, farm and home development 
families) but in statistical considerations they can 
be virtually anything (trees, fruit flies, guinea pigs, 
supermarkets, lumps of coal, fish, cans of corn). 
Thus, our population is that group of people among 
whom we are interested in noting behavior changes 
resulting from, or likely to result from, our educa- 
tional and organizational methods. 

The educational or organizational objectives will 
determine which people are to be included in the 
population for a particular study. An objective 
aimed at finding out what farmers learned about 
dairy herd improvement practices from our teach- 
ing efforts should be evaluated by surveying a 
population consisting of those farmers who have 
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dairy herds, or expect to have dairy herds,, and 
who were exposed to^the learning situation.' 

An objective dealing with teaching homemakers 
better child nutrition practices should be evaluated 
. in terms of a population crnsisting of those women 
who have, or expect to have, children to feed. 
Again, some criterion of exposure must be included. 
An organizational objective to set up a county 
extension council of local 4-H leaders has a poten- 
tial population of all the local 4-H leaders in the 
county. 

The population should be identified at the time 
the plan of. work, or the organizational plan, is. 
made up. If it was not done then, it must be^done 
before the sample can be drawn. 

The educational or organizational methods used 
in attempting to accomplish the objectives usually 
limit the population to a select group within the 
specialized population, as well as within the total 
population or thfe area. Although it is desirable 
to reach all dairy farmers, or all homerir§lcers with 
children, or all local 4-H leaders, it is often impos- 
sible to do so.' Only those who are- reachable by 
means of the methods used should be included in 
a population we plan to sample to provide evidence 
■for evaluation. 4-H Club , members make up the 
population being reached through 4-H Club teach- 
ing methods. Only the dairy project members are 
reached through dairy projfect work. 

Before some of you take exception to the last two 
statements, we will pause to consider why they were 
stated so positively. For our immediate purposes, 
v/e have to ignore secondari^ exposure. It is always 
possible that there is a larger population than -the 
one which we can define in terms of our objectives 
and -those most likely to be directly associa'ted in 
the 'learning experience. This larger population 
may include friends and neighbors of the (Jefined 
population; it can include other members 6f the 
family and casual passers by. 

However, it is so difficult to draw this larger 
boundary that we are forced to ignore it for pur- 
poses of evaluation. If attempted, errors of defini- 
tion could result in defective analysis and erroneous 
conclusions. If we feel we must know the identity 
of this larger population we can set up a separate 
suwey or study for that purpose. It is much better 
toioferate the delayand drain on resources to make 
such a survey .than to hazard a guess based on ^the 
sketchy data we may have at hand. 

In your program planning you decide whom you 
would like to reach or teach. By the methods you 
decide to use, you identify those you should reach/ 
or teach. Jhen, what you do as a teacher or organ- 
izer, a'nd how you do it, determine- the population 
from whom you can collect evidence for evaluation. 
Who is in the population, and where they are, \wll 
determine what methods you must employ to col- 



lect the evidence you will need to evaluate your 
program. 

The size of the population will help lis decide 
whether or not a sample should be used. If it is 
practical to contact the entire population, there is 
no need to resort to a sample, which always intro- 
duces some element of error, regardless of how 
well it is handled. When we deal with the whole 
population we do not have to concern ourselves 
with matters of testing for the statistical signifi- 
cance of our, results, or the representativeness of 
our sample. 

This suggests the next term we should discuss, 
representativeness. This is the essential ingredient 
in the last half of the definition we suggested at 
the start of this chapter. It requires that the sample 
from whom we get our information ^ust be repre- 
sentative of the whole population. If our sample 
is to serve as a reasonably accurate estimate of the 
group, ij/must po^ss all the charadtferistics likely 
to be^ncounterm in the group.^^he degree to < 
which it fails ti meet this criterion will be the 
degree to which X is likely to fail as a spokesman 
for the group, and the degree to which our conclu- 
sions based on the sample data may be in error. 

The way you select your sample will determine 
to a large degree the reliability, of your evaluation 
or study. A device, methpd, or technique used in 
sampling is reliable only to the degree to which thr 
results you get by its use approximate true condi- 
tions. You may be convinced that at least some of 
the people in your county actually were changed 
by your teaching effort. . If your sampling tech- 
niques are sound, the percentage of people in your 
sample that changed practices should be very near 
the true percentage of all the people who changed 
/ practices. 

/We can do two things which will contribute to 
. the accuracy with which our sample approximates 
true conditions: (1) We can see that a sample of 
appropriate size is drawn; and (2) we can arrange 
for appropriate precautions to assure adequate 
representativeness. 



Sample Size 

A common error in selecting samples is to take 
refuge in sheer size of the sample, as though sample 
size by itself can assure representativeness. Actur 
ally, size alone is no prooi of goodness or represent- 
ativeness. If the sample is not representative to 
begin with, size does not help at all. A sample of 
1,000, improperly selected, can be* less representa- 
tive, and can provide less reliable data, than one 
of 100, properly selected. 

The size of the population is only parMallyJndi- 
cative of the proper size for the sample . The im- 
portant factor for deciding on size of tb * sample is 
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the presumed homogenicty of the people within the 
population. It is obvious, then, that describing 
sample size as a certain percentage or'proport^ion 
of the population is not a realistic clue to adequacy 
. of sample size. In some situations, a 5-percent 
sample of a small population may be too small; in 
others, a 5-percent sample of a much larger popula- 
tion may be adequate, or even unnecessarily large.;. 

Proper sample size for the particular evaluation 
or study depends on: (1) Desired level of statistical 
accuracy, (2) costs in material and other resources, 
(3) homogenietjft.01 the population, and (4) ftbntem- 
plated intensit}%f analysis. ^ 

Whenever a sample of the population rather than 
the whole population is studied, the results have 
what we call an '*error/' ^'standard error/' or 
**probable, error/' • When used as we are using it 
here, this **error" d&es not imply that a mistake 
has been m?ide. Rather, it serves as an estimate 
of the probable limits of difference between any 
result we get and the- true result. The larger the 
sample \ye use, the smaller this error factor be- 
comes. 
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Figure 1. — Diagram showing range within which snmpio 
percentage may fall if true percentage from total 
population is 50 — by sample size. 
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In figure 1, the shaded area show? the range 
3vithin which a percentage obtained through study 
of a sample is most likely to fall if the true percent- 
age for the population from which the sample is 
drawn* is 50. If the sample size is 25, and the true 
percentage is 50, the percent;\ge obtained from a 
sample might be anything from 30 to 70. In other 
- words, the error might be as great as 20 percentage 
joints in either direction fro(p the true figure. 

With a sample of 100, the percentage obtained 
from a saftnple might fafl between 40 and 60 fa 
possible error of 10 percentage * points). If the. 
sarr'ple is increased to 500, the percentage obtained 
almost always would be expected to fall within 4.5 
^percentage points of the true percentage. 
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SAMPLE SIZE 

Figure 2.— Diagram showing range within which sample 
percentage may fall if true percentage from total 
population is either 90 or 10 — by sample size. 

TheTiearer the true population percentage comes 
to the extremes, 100 or 0 percent, the narrower 
becomes the range of probable error in the percent- 
age obtained from a sample of the population. If 
sample size is 25, and the true percentage is 90, the 
percentage obtained from the^ sample will most 
likely fall between 78 and 100. * 

If the true percentage is 10, the percentage ob- 
. tained from a sample' will most likely fall between 

0 and 22. This means that the error might be as 
large as 12 perijent. Referring back to figure 1, 
it will be noted that for the same size sample (25), 
the possible .error is 20 percentage points when the 
true percentage figure is* 50. 

We can conclude that the nearer the population 
percentage is to 50, the nearer the probable error 
range will be to that depicted in figure 1; the nearer , 
the population percentage is to either extreme, the 
hearer the probable error range will be to that 
' depicted in figure 2. Regardless of the size of the 
true percentage, however, the larger the sample 
size is, the smaller will be the probable error. 
(Observe how the range narrows on each Of the 
figures as larger samplfe sizes are used.) > 

This consideration of the probable error factor 
suggests that we cannot predict exact numbers or 
percentages for our population as a result of our 
observations of a^ sample. However, in much of 
our work, exact numbers are not vital. We are 
not so concerned with the exact number or percent- 
age of people who have made some desired change 
in beWavior as we are in learning whether or not a 
relatively large or a comparatively small amount 

01 change occurred. 
The cost factor includes time and labor and other 

resource uses, as well as cash outlay. Since this 
factor looms so conspicuously in any research or 
evaluation plan, we must work with the smallest 
sample we can and still feel confident of getting a 
usably small margin of statistical error This de- 
mands that we be especially carefL\l to make our 



sample as s'tatislically representative as possible 
within our size limitations. 

Regardless of resource limitations, the sampTc 
must be large enough to provideHhe opportunity 
'for all characteristics we desire to" analyze lo be 
in the sample, or representativeness will be sacri- 
ficed and the accuracy of our conclusions' \\J'ill be 
subject to question. If age, education, farming 
status, nationality, distance from town, and other 
selected factors are presumed to have bearing on 
desired behavior change, then the sample musjt be 
' adequate to assure a strong'likelihood that most of 
the^se characteristics will be well enough represent- 
ed to permit a reasonable amount of subsorting 
on^each of them. This means we would want most 
of the age groups represented, most of the farming 
status groups represented, and so on. ^ ^ 

Furthermore, we would want reasonable assur- 
ance that any special groups we desire to examine 
' more intensivoly will have a sufficiently large num- 
ber present su that results of our analyses can' be 
rQlied upon. A category containing less than 30 
items will have such a large probable error factor 
that the findings related to it will have very lltt4^r- 
if any, predictive \alue. For most of our work in 
E.xtension, a homogeneous group of 100 items w-iil 
carry an error factor small enough to allow for 
reasonably confident applications of the findings. 

If we -should desire to sort our category contain- 
ing 100 items into subcategories (sorting the re- 
spondents in^the selected age group into several 
level'of-edi^ation. nationality, or farming status 
categories) we w^ould have to use a larger sample 
so tl\at the statistical error for each category is 
^within usable limits. 

Such decisions regarding the amount of subsort- 
ing we plan to do must be made before we draw 
our sample if we want usable data. We' wilFfind 
that simultaneous sorting on more than two char- 
acteristics will usually require a prohibitively large 
and expensive scimple, unless the population is an 
exceptionally homogeneous one. However, we can \ 
make tentative analyses using several sorting fact- 
ors* if w^e hold down the number of categories in a 
classification* (older members — younger members 
rather than 0-14, 15-19, 20-34, and so on, age 
groups). 

- The RepresehiaJive Sample ' 

X ■ 

Randonr sampling means selecting a sample m 
such a way that each item or persqn in the popula- 
tion being studied will have an' equal chanre to be 
selected. A sample chosen by random selection is 
more likely to have the different charasteristics of 
the population, and in the same proportions, than 
any other sample you can select.^ If a sample chosen 
bv this method also has adequate size, it is n\ore 
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likely than^ot to resemble the poptilation from- . 
which it reelected.. 

Previously, we have been avoiding the use of 
^'selection'' in speaking of sampling. Now that we 
begin using it, we must^ make clear that we are 
using it in a statistical sense. Actually we do not 
select the sample. What -we do is decide on the 
method of selection and leave* final selection to 
the method. 

If the method says we^'take every fifth name from 
a list, starting, with the second name, that is what 
we do.' \Ve donH start with the first name on the 
list, or take^.the sixth or fourth ndme once in a while 
just for variety. We do not substitute or deliber- 
ately omi^a name. 

Thereymay be occasions when such things .are 
permitted, but they must be covered in the rules 
agreed upon before selecting the Sample. What- 
ever the justification for their use, they impair the 
randomness and representativeness of the sample 
and should be- discouraged, unles's there are com- 
pensating reasons far shifting from a strictly ran- 
dom sample to a modified rai^dom, or purposive, 
stimple.,^ 

The primary objective of random sampling is to 
assure equal opportunities for all units to be 
includfid. It is not enough to know that certain 
biasing factors have not been intro4uced at -the 
point of sample selection;. it is also necessary to 
know that all parts of the population have a chance 
of being included. 

Our aim in drawing a sample that ^vill serve its 
purpose in evaluation is tq^ave one that will repre- 
sent the population from which it is drawn^ Ac- 
cording to Earl Houseman of the Agricultural 
Marketing Seryice, *'A representative sample is a 
probability sample such that the sampling errors 
and biases associated with the method of selection, 
nonresponse, and estimation are known to b^e suffi- 
ciently lo^v to serve a useful purpose."*' 

Several methods of dj^wing samples^by, random 
selection have^een used in extension evaluation 
work. We ^vill discuss several and mention the 
known limitations they place on true representa- 
tiveness, along with the reasons which help justify 
their use. 

List Sampling — One method of sampling is to 
select a group of people from a list. Through the 
use of rahdom selection methods, every person on. 
thd list will have just as good a chance of being 
selected as another. The list must, of course, con- 
tain names of the total population in which we are 
interested. This method should be used whenever 
a complete list of the total population is available. 

There are two favorite ways of selecting a ran- 
dom sample from a list of names. For either 
method, the names should be arrayed in some man- 
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ner and nirmbered. It makes' no difference how 
they are arrayed before they are numbered, such 
as alphabetically, geographically, by mailing ad- 
dresses, or by clubs. 

The first, and easiest, way is to select evof5"<;Tith" 
name from tho list. If you need one-foufth of^he 
names on the list, you select every fourth nartie; 
if you need one-twentieth of the nqmes to make an 
adequate sample, you select every twentieth name\ 
To calculate the proportion of the names thay 
ypu will require (to determine what the *'n'* w^itl 
be), divide the number in your population by the 
number you desire for your sample. Convert this 
to a. fraction with a "1" for the numerator. Ex- 
ample: There are 7,697 names on the list. A sample 
of 350 is desired. You get approximately 22 when 
yoa divide 7,697 by 350. Therefore, your fraction 
is 1/22 and you will need to select every 22nd name. 

'To determine your starting point, select a num- 
ber by some random method from 1 throygh 22. 
Use this number as the first one in your sample and 
take every 22nd number from that point to the .. end 
of the list. . " 

This method meets the criteri^on of equal chance 
as long as the original numbecis selected in a purely 
random fashion. We can resort to a table of ran- 
dom numbers to determine the first number, or >ve 
can place the numbers on separate cards or slips 
of paper and draw one from a hat. 

Sometimes we can pick this starting number in 
some way which might violate slightly our repre- 
sentativeness ideal, but which will have a symbolic 
implication for the study. We might use the day 
of the month, the month of the year, or some other 
date-baseji starting point related to the starting date 
of the study. We might use the hour or minute al 
the time the starting number was picked. 

Suppose we sit down at 10:45 it the morning 
with our completed list before us and with our 
sampling fraction calculated. ^Rather than fumble 
around with preparing slips for a drawing, we 
night get on with our w^ork by arbitrarily selecting 
*10'' as our starting number (45 would be too large 
"or our purposes). Then the 10th name woiild be 
he first in our sample, the 32nd {10 plus 22) would 
)e the second, and so on. 

Another method of random flection from a list 
s using a random numbers tab!&. These nuniLers 
lavc been assembled into tablesjby purely random 
neans. At the time they were put up in tables, 
?ach number had an equal chance of falling in any 
position in the tables. 

To use such a table, you start anywhere on it and 
2ke each number which falls within the range of 
umbe.'-£ in your population until you have enough 
or your sample. In our example, with a popula- 
ioa of 7,697, we would use the first four columns 
1 each row, because of our 4-digit population total, " 



and include every number less than 7,698 that we 
come to until w^e have 350 numbers. ^ (For- more 
information about using random numbers tables,- 
see reference 9.) 

For long lists of names, this method is very time 
consuming. However, it' is a more reliable method 
than selecting every **nth" name if there is -likely 
to be any cyclic or patterning effect in the original 
listing. Conceivably, you could wind up with all 
males or all females if you were to draw every **nth" 
narrie for a 25- or 50-percent sample of a member- 
•ship list of a young couple's home planning club. 

Area Sampling — More often than not, you will 
find it necessary to work wit:iout a complete list of 
the population. In order to sample a population 
- whose individuals are not readily identifiable, the 
system called ''area sampling" has been devised. 
In area sampling you cho'^se a random sample of 
geographic areas instei : ! random sarhpling of 
individuals. You base yv>,. > v .at!y;ds on data secur- 
y. , ed from all the individuals who live within the 
selected areas, and who qualify according to your 
definition for inclusion in the population. 

To set up an area sample a map of the area con- 
cerned is marked off into segments which have in 
them, on the average, a predetermined number of 
units (farms, households,, dwelling units). When- 
ever possible, boundaries that are easily seen while 
driving along in an automobile (township lines, rail- 
road tracks, highways, streams) should be used to 
bound the sampling areas. When an area is so 
thickly settled that it is impossible to use easily 
tHentifiable boundaries, aerial' photographs which 
reveal more minute details of the landscape (fields, 
farmyards, gardens, orchards) can be used. 

Before starting to mark off sampling areas, you 
. need to decide the number of units you wish Jjp 
have in each area. This number may be of amy 
convenient size, depending on situational and study 
requirements (homogeniety of the population, de- 
sired statistical accuracy of results, contemplated 
detail of analysis, convenience in interviewing). 

Next, you start in the upper right-hand corner 
of the map and count off a number of units around 
which you can spot identifiable bouhdaries. If the 
number is over half as large as, 'or is less thanTialf 
again as large as, the deshed average number, it 
can be considered a sampling area. If the number 
is much smaller than the desired average, you com- 
bine it with the next one you outline. If the num- 
ber is much larger than the desired average, you 
divide it through some means other than identi- 
fiable boundaries. 

Move across the map to the left-hand side, mark- 
ing off areas as yoy^come to them:. Then move back 
'-'"across the map to the right-hand edge. Continue 
this process until the complete map is marked oft 



into sampling areas. Next, number these areas 
consecutively, again starting, in the upper right- 
hand corner of the map and moving to the left, then 
toward the right, and so on, in a serpentine fashion. 
When all areas *ire numbered, ^i^you .sample from 
them just like you would from a numbered list of 
names. Sampling can be carried out by cither 
method described under "List Sampling,'' although 
the fii'st method is most often used. 

In those parts of the country where the grid 
system of surveying is used in the rural areas, the 
section and township lines provide easily identi- 
fiable boundaries for area sampling segments. In 
rural aieas in other parts of the country, and in the 
more densely populated areas, plat m.aps and aerial 
photographs are used to advantage. You will 'find 
that the major problem at this point is getting 
materials which were prepared recently enough so 
that you do not have to take time to bring them up 
to date before you can use them. 

An important advantage in using area sampling 
is that your respondents are concentrated into 
groups by the manner of selection. This holds down 
costs associated with travel, maintenance, and 
supervision of field personnel during the study. , 

A disadvantage is that, in order to hold the statis- 
tical error down to a usable point, a larger total 
number of people will need to be included in the 
sample. The geographic areas themselves are con- 
sidered the sampling units.for determining statis- 
tical error. For example, if 65 geographic areas 
which include 325 respondents are covered in a 
survey, 65 is considered the sample size, and not ^ 
the 325. ' • ^ 

Grid or Random Point Sampling — Another 
method of sampling geographic areas is to use a 
**spot'' method. One way is to lay a grid oyer your 
map and choose the units located at or nearest to 
the points where the lines cross* You might also 
use the segments marked off by the grid as areas, 
number them, and select from among them by 
using a table of random numbers or some more 
arbitrary way. You will run into a problem of 
identification here,- however, as you will with any 
system^ which does not make use of conspicuous 
natural or man-made landmarks. 

Yet another way is to put a number of dots, hit 
or miss, over a piece of transparent paper and lay 
it over the map. You can use these points as cen- 
ters of sample areas of predetermined size, or you 
can take the **n'' units living closest to this point 
to be your respondents. 

Eachzscjf rthese methods can be deficient in repre- 
sentativeooss if the population is IjLkely to be spotty ^ 
in the way in which characteristics are distributed* 
throughout the area. However, there is a known 
probability ratio for being included at each stage 



o/ the screening process, so they qualify on strictly 
statistical grounds. 

Group Sampling — Sampling groups of people 
instead of {individuals can be done by using' the 
same principles that are' used in area sampling. In 
extension files we have all kinds of lists by clubs, 
both membership and mailing lists. We can use 
the mailing lists to classify people into the geo- 
graphic areas (such as townships) in which they 
live. Then we can interview all tlie people who live 
in designated areas, instead of the entire member- 
ship. 

If this still leaves us with too large a sample to 
question, we can sample within the areas by re- 
assembling the lists by areas and drawing a frac- 
tional sample from each in the manner suggested 
earlier for straight list sampling. This is an ex- 
ample of double-stage sampling.^" Other stages 
can be set up, and often are, but they add immerise- 
ly to the complexit}' of the study and should be held 
down as much as possible. 

■ Using the group sampling - technique, we can 
study club effectiveness by studying representative 
clubs, or club members located in representative 
areas of the county or State. The technique works 
equally well for checking on acceptance rates of 
recommended farm practices and for observing 
most kinds of behavior change in relation to the 
extension teaching practices used. 

•Unfortunately-y this technique increases statistical"* 
error, but it decreases cost. If we have reason to 
believe that differences are as great within groups 
as between them or, conversely, t^'at one group is 
apt to be very much like another m regard to the 
matter under consideration, we can use larger 
groups for sampling units, thus saving even more 
time and other resources. 

As with area sampling, it costs mor^e when a 
sample of 5 individuals is chosen at ranqom from 
each of 20 groups chosen at random (yielding a 
sample of 100). However, the resulting sample pro- 
vides more statistic:ally a4:curate data than does a 
sample of 10 individuals chosen at random from 
each of 10 groups chosen at random (also yielding/ 
a sample of 100). 

Stratified Sampling— Thus far we have described 
sampling methods aimed at bringing into the sample 
all characteristics of the people in the same propor- 
tions as in the total population. However, if the 
sample is small, it is possible that characteristics 
belonging to small groups within the population 
may be omitted. In order to be sure that certain 
especially important characteristics get included in 
the sample, some method of stratification is recom- 
mended. 

Stratified sampling involves dividing the total 
population into several separate populations in 



such a way that these populations are different in 
regard to selected known cliaracteristics (age, farm- 
ing status, size of cityj. If the purpose, of using a 
stratified sample is to assure that certain character- 
istics are included in the sample in proper propor- 
tion, the same sampling fraction is used for each 
stratum* (each separate population). 

If the purpose of using a stratified sample is to 
obtain records from pnough. individuals having 
specified degrees of certain characteristics so that 
these subtypes can be studied separately, a differ- 
ent sampling fraction can be used for each stratum. 
Whereas one-twentieth of one o£ the strata may pro- 
vide an adequate size sample, it may require one- 
third of another stratum to provide the same size 
sample. 

This technique is used very frequently in studies 
involving farm operatois. We realize how widely 
farm operations vary in size, and we also realize 
that behavior of farm operators on the different 
sized holdings is likely to differ as widely. To be 
sure of adequate numbers of respondents with 
operations in the different size categories, we set • 
up several different sampling fractions. We might 
have to use all the very largest operators in the 
area (1/1 fraction). A 1/10 fraction might do for 
operators with holdings in the middle range. 
Among the much more numerous small or subsist- 
ence operators, a I/106 fraction might be adequate. 

When this method is used, data from the different 
s.trata cannot be assembled by simple addition for 
purposes of obtaining totals for the population. 
Data from each stratum first must be weighted 
according to sampling fraction dsed. In our farm 
operator sample, we would have to invert the frac- 
tion for each stratum, convert it to a whole number, 
and multiply by this term, before adding up to get 
population totals. We would count our large oper- 
ator data as observed, but we would have to multi- 
ply our middle range operator data by 10 and our 
small operator data by 100 before totalling. 

Judgment Sampling 

.The sampling methods we have been describing 
depend on objective procedures which should re- 
sult in fairly representative samples (everyone in 
the population has an equal or known chance of 
beir^g selected). There are occasions, however, 
when you might find it better for your purposes to 
choose a sample on the basis of what you believe it 
should ^'ontain. In so doing, you hand-pick each 
individual to go into the sample, deliberately mak- 
ing up what you feel is a sample representative of 
the population from which it is cho;^on. 

Yates, the English statistical authority,^ has this 
to say about this method: 

*'The ideal which is aimed at in sampling is to 
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make the sample as representative as possible, so 
that measurements or observations on it cart be 
taken as virtually equivalent to similar measure- 
ments on the whole population. The fact that this 
ideal is in the mind of the sampler when taking the 
sample naturally influences his selection if he has 
any freedom of choice. Most samplers w^hen select- 
ing a repref-entative sample will deliberately reject 
abnormal material, or -if they feel, that the sample 
should be^representative of the abnormal as well as 
the normal will deliberately balance up the differ- 
ent categories abnormally. 

^'Unfortunately the sanipler's claims to be able 
to select *a 'representative sample' by personal 
judgment are largely unfounded, and his selection 
is in fact subject to r . orts of biases, psychological 
and physical. To avoia these biases and to provide' 
an estimate of the representativeness of the sample, 
i.e., of the 'sampling error', more rigorous processes 
of selection has been devised."^'- 

Houseman^comments: ''It is reasonable to expect 
that judgment samples will usually be selective in 
, the direction of what the mind regarcls as the more 
important elements or in the direction of the less 
obscure elements. It is also reasonable to expect 
in general that the variability would be^less within 
judgment samples than within the whole popula- 
tion; if so, this would mean that distributions based 
on judgment samples might be seriously distorted 
, . . Judgment samples naturally look good to the 
person who made the selection, but since a sample 
cannot be appraised by its-looks, judgment samplers 
are likely to have erroneous, impressions of the . 
accuracy of their samples/'' 

Limiting the Population to Study Size 

Within certain populations, especially when vari- 
ability is great, it may be advisable to define the 
population in such detail that all or nearly all of 
the merribers of the population which have the 
desired Characteristics riiust be included in the 
survey, ffn such cases, sampling, as defined ]f\ this 
chapter, (is neither necessary nor appropriate. If- 
you should wish to study 14-year-old, Ist-year 4-H 
Club mernbers in a county, you would seldom find 
enough of them to -make up a population large 
enough to sample. You would have to use all of 
them. 

We must remind ourselves constantly that, as a 
result of narrowing the population to study size, 
we can apply the findings only to the group we 
have studied. Comparisons with similar 'groups of 
14-year-old, Ist-year 4-H Club members in other 
counties may be made, but only with due allowances 
lur differences in environmental elements of the 
different locales. We can draw no direct inferences 
to another or larger group or population. Anyone 
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who .desires to use such data for purposes of com- 
parison must be made aware of these limitations, 
and should use them, if at all, only after careful 
consideration and interpretation. 

Testing the Sample 

We have statistical formulas for determining 
whether or not a sample is of adequate size to 
accomplish the objective for which it is drawn. We 
also have checks on representativeness. We can 
use the following methods quite effectively • in^ 
extension evaluation. 

If the population to be studied is a class of people 
for whom information is available from census 
.sources, we can check data from the study against 
comparable censUs data. Sometimes, when infor- 
mation requested for census purposes may not be 
relevant to our particular evaluative effort, we may 
,ask a few census-type questions, anyway, so that we 
can have data to check with census materials to 
get a line on the representativeness of our sample. 

Another test for adequacy is to draw from the 
.population several other samples of approximately 
the same size as the given sample. .Then we can 
compare the results for the different samples. Un- 
fortunately, this can become prohibitively expen- 
sive. A more economical technique with the sarne- 
objective is to'divide the sample into two parts, and 
compare^the results. 

^ However, even when such are .used, they do not 
serve as a complete check on representativeness. 
' Of greatest importance is the assurance that qx-, 
trenie care has been taken in following the sample 
selection methocf, in obtaining information from 
the total sample, and in collecting information in 
such a way that biases and errors are minimized. 

Summary 

Most evaluation problems require sampling, in 
some form. For that reason, we have devoted this 
cliapter in our methodology section to a discussion 
of what sampling is and ho\V it works. ' 

After some basic definitions and a rundown of 
the fogr steps involved in a typical sampling.prob- 
lemr we discussed" several ways to san^ple arid men- *' 
tioned some^of the conveniences and shortcomings 



of each-. We included some of the more approved 
methods, as well as several which may have to be 
used when conditions will not permit an entirely 
objective sampling, plan. We stressed^ the import- 
ance of size and representativeness in a sam'ple and 
suggested some ways to .check on each. 

We discussed how probability or chance enters^ 
into the rationale of inferring from a sample to a 
population. Without going into the mechanics of. 
statistical computation, we commented on tlje im- 
portance of 'using statistical tests for probable 
error and reliability.-^ 
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Chapter XI 
METHOQS. OF COLLECTING DATA 



Up to this point we have been discussing the 
objectives of the evaluation, evidence of the objec- 
tives, and tlie persons or population from whom or 
about whom we will collect the evidence. In this 
chapter we will discuss methods for collecting the 
evidence or data.^ 

The method of collecting the data should nevqr 
determine what we collect. The most in>portant 
point is to be sure of exa -tly what information we 
need iCf meet the require iuents of the study. The' 
person conducting the evaluation then needs to 
decide what is the most practical way to collect the 
data. 'He heeds to consider the type of device he 
will need to get a record of evidence.. He will also 
also need to consider the conditions under which 
he will have to work, and other demands on his 
time, budget and other resources. 

Methods of Collecting Data 



usuallv used 



1. Mail questionnaires. 

2. Personal-interview schedules, 
while interviewing people in their homes. ^ 

3. Distributed questionnaires and checklists -of 
all kinds.- 

4. Group interviews. - ' ' . 

5. Case studies. 

6. Systematic observation procedures, 

7. Systematic studies of available records. 

8. Tape ' recorded interviews, discussions or 
panels. 

Considerations Which Influence 
Selection of Method 

1. The^olpjcctivcs of the study. 

2. The type of device that might be used to ob- 
tain evidence. - 

3. The respondents from whom you will need 
to get the data: 

a. Where and' how can they best be leached — 
at home, through .the mail, at meetings, or 
elsewhere? 

b. Will they represent a ijamp.lc- of the totVl 
population which the extension worker h^s 
attempted to reach? 

4. The resources you have available. 

a. Your time — demands of other work. 



b. The time of others wno will also work on 
the study. 

c. The time it vvill require of the respondent, 
the person who is to ans'wer.the questions 
or give the information. 

d. Money available for supplies, stenographic 
and statistical help, travel, subsistence. 

5. The advantage and limitations of each method. 

Analysis of Each Method 

In selecting the method to use in collecting data, 
the evaluator needs to analyze each method to see 
whether or not it will provide him with valid and 
reliable information. Each method has its advan- 
tages and limitations. These should be taken into 
consideration in making decisions. Let us first 
take up the mail questionnaire. 

The Mail Questionnaire — This is sent to respond- 
ents who fill it out afid return it by mail. Before 
deciding to use this method, there are certain ques- 
tions'^ which we need to consider. 

1. Can the questions on the objectives of study 
be fitted to a mail questionnaire? 

2. Can the question? be stated in a clear and 
understandable fashion without further ex- 
planation? ' 

3. What about the people returning them; arc 
they representative of the group with whom 
we have been working or of the ^'population''? 

4. What about response rates? 

Advantages' of the mail questionnaire: 

1. It is an easy, quick and relatively inexpensive 
method of obtaining data, especially when 
compared to a personal interview designed to 
secure the same information. 

2. It can be planned fo reach a wide geographic 
distribution of people including some quite 
inaccessible people, provided that their ad- 
dresses are known; 

3. It is useful in reaching a relatively homogen- 
eous, fairly well-educated group. 

4. It is well adapted ior reaching specific groups 
such as dairy farmers, poultryn>en, or home- 
makers who belong to organi'/ed home domon- 

. stration groups. 
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5. It can be free of any interviewer bias, but not 
necessarily instrument bias 

Limitations of the mail questionnaire: 

1. The questionnaire must be short. 

2. It is difficult to obtain detailed qualitative 
answers or to know precisely what the re- 
sponses mean, . ' 

3. There js little control over completeness of 
answers; when the iespondents don't under* 
stand or don't want to say, they may just 
leave out the answers to a question. 

4. Those who reply may not be typical members 
of the list; those who have made changes in 
practices or who are especially interested or 
who are especially opposed to the ideas pre- 
sented are the more likely to reply. 

5. Questionnaires should *be returned from 
everyone in the sample. This usually requires 
followup. It takes time to write followup let- 

^ ters, to make telephone , calls and to call up 
respondents who fail to reply. ^ 

Personal Interview Schedules — These .schedules 
:rire filled in by an enumerator or repcrt^M*. This 
(method is used in a more formal type of .study in 
which an interviewer calls on a sample of people. 
The interviewers are usually neighboring county 
extension workers. State extension workers, and 
lay people, such as extension committee members. 

Advantages of personal interview method: 

1. The people who participate have an opportun- 
ity to observe and study situations and condi- 
tions: they talk directly with people and get 
their reactions to practices and programs. 

2. The personal interview method usually yields 
a high percentiige of returns, as niost people 
are willing to cooperate. 

3. The interviewer has an opportunity to explain 
questio^^s to respondent. 

4. Complole answers to all questions can usually 
be obtained. This contributed to statistical 
accuracy, validity and reliability. 

Limitations of the personal interview method: 

1. Transportation costs^and time required in a 
huge area may make the personal interview 
method impracticable. For example, good 
enumerators, unless volunteers, can demand 
very good salaries. An enumerator might be 
tied up several hours on a single interview. 

2. Unless the interviewers are properly trained 
and supervised, the data recorded may be in- • 
accurate and incomplete. A few poor enum- 
erators ran bias the study. 

3. It is usual l> claimed that costs per interview 
ai'c higher than when ntail questionnaires are 
used. This may not be true if the area to be 
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covered is not too great. The personal follow- 
up of the jnail questionnaire in order to secure 
records needed may run the cost as- high as 
for pc.sonal interviews. 

Distributed Questionnaires or Checklists — ^These 
instruments are often used interchangeably. They 
arc usually handed directly to the respondents in- 
dividually or in groups. Respondents answer the 
questions and hand them back to the person con- 
ducting the evaluation. 

Respondents are asked to record th'bir answers 
to questions by checking or otherwise indicating 
their selection from a list of statements. These 
arc usually distributed: 

1. To all persons in a group or in attendance at 
a meeting, to be filled out at the meeting. 

2. To a sample of people who are participants in 
a group or meeting, but to be filled out at 
home. ^ . ^^^^^ 

3. To leaders whoj'ask their neighbors or mem- 
bers of the group for information. 

These small report forms are used ext^isively 
in home demonstration work. They are used to 
collect- evidence of progress made — usually prac- 
tices adopted — for use in program planning aM in 
writing the annual report. 

Advantages of the^distributed questionnaire: 

1. The questions have a definite relation to rec- 
ognized goals or objectives. 

2. The cost is small and they are easy to admin- 
ister. 

3. They are, easy to complete. 

4.. They provide a quick way of surveying the 
group. 

5. When 'used before and after a meeting or a 
series of meetings, they can be used as in* 
terest gutters. 

6. If. filled out in a meeting, they cairbe used as 
a basis for discussion and can indicate the 
next steps in program planning, (People in- 
volved in filling out the forms are also usually 
involved in program planning process.) 

7. They can also be used as attention holders and 
indicators of action. 

Limitations of the distributed questionnaire: 

1. The people who fill out these forms are usual- " 
ly participants in the program and know the 
objectives and may be inclined not to be 
critical of the existing program, 

2. These report forms may center attention on 
'^practices adopted'' rather than on other 
kinds of behavioral changes. 

3. Participants in groups in an area are not 
usually representative of all people in the 
area. Therefore, it is better to inter\^ow a 
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cross section of people in the area if we desire 
to measure changes made by others than those 
in the organized group. In other words; these 
individually distributed questionnaires -tend 
to measure progress only of those who partici- 
pate in the program. 
4. They are usually on single topics and there- 
fore cover only a limited field of study. 

Group Interviews — ^The leader of the study pre- 
sents a questionnaire to a group of people assembl- 
ed in one place. Each person in the group is asked 
to record his or her answers to the question as it 
is read. To secure an adequate cross section, those 
people interviewed must be carefully selected so 
as to secure representative samples of the people 
about which generalizations are to be made. 

Advantages of group interviews: (Advantages are 
similar to those of the individually distributed 
questionnaires.) 

1. If the persons in the group are homogeneous 
and if the persons attending the meeting are 
representative, data are easily and readily 
obtained. . 

2. There is very little cost — easy to administer, 
very little travel, a small crew can do the job. 

3. Speedy — the whole interview can be conduct- 
ed in a very brief time. 

4. There will be few refusals. 

5. The questions can be used as a basis for dis- 
cussion and can help toward program plan- 
ning, as well as to hold or get interest. 

Limitations of group interviews: (Limitations are 
similar to those of the individually distributed ques- 
tionnaires.) 

L The replies may be unconsciously or other- 
wise influenced by those with strong opinions 
jpoer group bias). 'Since you want indepen- 
dent judgmofits, discussion should be discour- 
aged during the interview. 

2. The people attending a particular meeting are 
seldom representative of the entire group. 

3. The questionfiaire used for group interviewing 
is usually short and therefore c^n cover only 
a limited field of study. 

Case Sfiidies — These can be .stuclies of one or a 
few individuals or cases, g chronological report of 
the development of a project or activity, or a story 
of an individual family. Frequently they are re- 
ports of concrete events which are vivid, convincing 
stories of the effects of a program. They may con- 
stitute important evidence, if properly assembled. 

Advantages of case studies: 

1. They give concretencss and renlity to prob- 
lems and solutions. 

2. They present the situation, reveal thi import- 
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ance of human factors in organization, show 
the difficulty of problems as they evolve. 

3. Often a case s^tudy will bring out important 
factors which- are ^assumed to be general 
knowledge and for tl\at reason go unmention- 
ed by respondents 1 interviewed by other 
methods. 

4. They may show how decisions are made. 

5. They show the importance of timing. 

6. They show the continuity of development and 
how the project or activity unfolds. 

7. They bring out the problems as .they arise, 
how these problems were solved, and how the 
mistakes were made. 

8. They afford an opportunity to study a process 
minutely and to discover its strengths and 
weaknesses. 

9. They are effective in serving as springboards 
to further thought and consideration — a good 
teaching device. 

10. They show Extension's contribution in help- 
ing to solve problems. 

Limitations of case studies: 

1. Case studies may give particularized data. 

they may emphasize the unique. 
2» They may provide data on only 1 or 2'^$pects 

of a problem, and not on all relevan. aspects. 

3. A case^ study is only one ^c^se; unwarranted 
generalizations may be made. 

4. They are time consuming and require training 
and experience in observing, recording and 
writing. 

Systematic Observation Procedure — This meth,>d 
is used during visits to homes or .farms. Observa-. 
tion of practices which have been recommended 
for the area are observed and recorded in some 
systematic manner. 

Advantages of systematic observation: 

1. The worker knows the recommended prac- 
tices. 

2. He can observe to what extent they are being 
adopted. 

3. He can have them listed on an observation 
sheet and check them off. This avoids hit or 
miss observation. ^ 

4. He gets firsthand information. 

5. He can obtain permanent record information 
at. the same time to supplement his observa- 
tions. 

6. Progress can be shown. 

Limitations of systematic observation: 

1. The observer may use his own values — attri- 
bute importance to what hp thinks is signifi- 
cant. An objective *checl< $heet helps to -^re- 
vent this. 



2. It is difficult to separate observation from in- 
terpretation. / 

3. Enumerator has only his intuition to toll him 
what is representative. 

4. Costly when much information must be col- 
lected. 

5. Impractical to do for much information. 

6. Details important for analysis may be omitted. 

Systematic Study of Available Records — 4-H Club 
project records and farm and home development 
records are frequent sources for this kind of analy- • 
sis. An analysis of .records and reports involves 
much more than recording, tabulating and report- 
ing. 

Advantages of records analysis: 

1. Data can be obtained directly from the rec- 
ords without going through a field survey or 
some form of interviewing. 

2. This may take the time of just^^one person. 

3. They are reliable if the records have been 
carefully kept. 

4/ They provide information fron^^the:pa«t whifh^ 
could not be collected today. 

Limitations of records analysis: 

1. Records may be carelessly kept and incom- 
plete. 

Records may not be froni a represenfaTivc 
sample of the populatioi^. 
3. Often there is no opportunity to make checks 
on accuracy of entries. 
. 4. Records may lack comparability and, 'there- 
fore, be difficult to analyze. 

Planning to Carry Out the jEvaluation 

We have discussed seven different methods of 
collecting data. Deciding on the data collecting 
method does not end our decision making action, 
however. \V(^ need to decide not only on the 
method of collecting data but also who will be re- 
sponsible for gathering the information, when the 
work will be done, aad where it will be done. 

Who Will Be Involved ip Gathering the Informa^ 
fion? — This varies with the type of information 
wanted and the method used in collecting data. If 
a mail questionnaire is to-be used, only the county 
agent may be involved. Howeve.^, lay people should 
be involved in -planning the evaluation and when- 
ever possible should, be used m collecting informa- 
tion. 

An important value resulting from i study is the 
training and exneiicncc gained by participants.^ 
The people who arc involved in the development 
and exemtion of a program should be involved iti 
its evaluation. These may bo; ^ \ 



1. Agent alone, or with other agents from home 
county, 

2. Agents from another county. 

3. Agents and lay people. 

4. Agents, lay people, and State staff personnel, 

5. Agents, lay people, State staff, and Federal 
staff personnel, 

, When Will the Information Be Collected?— It is 

desirable to decide considerably in advance the 
exact period that will be covered by^the study — the 
month or season, a specified time after the work 
has becii taught or the activity completed. The 
evaluation should be planned so as to avoid heavy 
work seasons of farm people. 

When the information is to be collected depends 
also on the other jobs fhat must be done in the 
evaluation and the time that must be allowed for 
each job/ Very early in our planning we should 
w^ofk out a detailed schedule spelling out, (1) jobs 
to be done, (2) when each job is to be done, and 
(3)^\;iio is responsible. 

Wh^rM^ill tfie Study Be Made?— Step-by-step 
development of plans have been oov&ced up to this 
point. The method used in collecting data, and 
from what people, will determine where the study 
is to be made. 

Summary 

The inethods of collecting data should not deter- 
mine what to collect. Information or evidence 
• needed to meet the requirements of the study is 
of first importance. The method used to obtain 
evidence depends on the kind of evidence you need 
and on the resources you have to (Collect it. 

Some methods of collecting, data are: Mail ques- 
tionnaires, personal-interview schedules, check lists 
of a?l kinds, group interviews, case studies, syste- 
matic observation procedures and studies of avail- 
able records. 
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Chapter XII 
INTERVIEWINQ 



Throwghout the process of developing a study 
plan w6 have to keep in mind how the Inrormation 
will be collected, in this chapter we will discuss 
interviewing as a. method of collecting data. A 
good interviewing job cannot salvage a poorly-dtr^ 
signed study but poor interviewing can lessen the 
value of a well-planned study. 

We need to know 'what kind of interviewers w^e 
will hay.e before w(? can decide how involved to 
make our plans for collecting data. 

We must know whether interviewers will serve 
without pay or will expect to be paid, and how 
much, before we can make estimates of costs for 
the study.' Wc nvjst know how many we will use 
before wocan estun^te l^ow long the data collecting 
- .phase is likely to take, and whether or not we will 
be^bl e to handle supervisory details without assist- 
ance. 

An interview is a.^onfact by an investigator with 
a person for the purpose of securing specific infor- 
mation* It can take place in a face-to-face getting 
with the interviewer asking questions and recording 
the answers he receives from the respondent. It 
can occur in a modified Tace-to-face setting where 
the interviewer asks for information from a group 
of. people. With this Arrangement, resppndents 
will record thojr answers in some.iYianner apd- the 
interviewer will be concerned, primarily, with clar- 
ifying statements and stAiving for completeness^ of 
respofiser^ ^ \ , 

• A third type of tontact\is by means of the tele- 
phone, wherein the inteiwiewcr solicits 'answers 
from the respondent via twe telephone and records 
thenron some sort of tallyNsheet or schedule. 

Most of us in Extension>kia good amount of 
interviewing at one or nioie of tfibsfi levels. Qften 
it is so casual that we do' not thinK\)f it as inter- 
viewing, and may g.ivc no sen9us thoirght.to evalu- 
ating the answers. Occasionally, though, we spot- 
check some phase of 'our work by interviewing 
members of our clientele, and^'oncc in a while wo 
. enter upon fulK blown evaluation problems requir- 
ing organizeci intervi(W'ing plans. 

In addition to suppling us with the information 
\\c need for our work, personal -inferviewing gives 
us a better understandijig of the people with whom 
we work and of* the community .in which x)ur work. 



lies. This is most true of the more personalized 
forms, but is proportionately so for all forms of 
interviewing, All too frequently, however, we do 
not give much thought about how to do it properly, 
or to recognize all the occasions when interviewing 
may be used to advantage. 

Major Kinds of Interviews 

Proper collection of data by means of interviews 
is a very important tool in research. 

As we discovered while discussing other tech- 
niques used in evaluation, those we can use fur 
interviewing also are varied, and equally subject 
to complicating factors and limitations. We will 
talk about the three major kinds of interviewing, 
that can be used to advantage in Extension and 
mention some of their advantages and disadvan- 
tages. 

Face-to-Face Interview — This is the most per- 
sonal level of interviewing and, because of the one- 
interviewer one^respondent pattern, it is the most 
w expensive. It is more time consufning to make the 
individual heme visits. Although there are occa- 
sions when it is possible to^ get the respondent to 
come to your office to be interviewed, most of the 
time yo u ha ve to go to him. 

Original transportation costs are greater per re- 
spondent, and several trips may be necessary before 
the prospective respondent is contacted and the 
interview is secured. There is also thj strain of 
repeating the processes of gaining admittance and 
establishing rapport (making the respondent want 
to cooperate). . 

To conipensafte for the heavy drain on resources 
required by personal interviews, there are several 
convincing advantages. The interviewer may be 
able to keep his respondent interested and atten- 
tive for a longer period of time by changing the 
pace or taking a conversational break whenever he 
observes signs of declining enthusiasm. This is 
(jifficult to do in a group situation or during a tele- 
phone interview. 

A personal visit provides the interyiSweNwith a 
chance to observe personal reactions/vvhich can be 
niuch more enlightening than the vcK^alized answer 
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alone. He is in a better position to note resistance 
and liiay be able to discover\reasons and deal with 
them mere easily in a personal sotting. 

Even though same people may respond more 
readily to a telephone interview, the interviewer 
has more opportunities to establish friendly rela- 
tions during a face-to-face visit. A smile, a com- 
ment on something visible, a personalized word of 
praise, an expectant attitude, all can be used by an 
alert interviewer to draw out an ansWer from a 
reluctant respondent. 

Telephone Inter-lew — The telephone interview 
may he very useful for certain purposes. Jf the 
interview schedule is relatively brief, we can assem- 
ble complete and accurate information in this 
rnanner. However, it is not so effective when 
"thought" questions are employed or when data 
which may. not be readily -available are required. In 
exceptional cases, and when skilled w^orkers are 
operating,- it is pcssible to keep a respondent inter- 
ested and attentive for a half hour or more. 

If the respondent happe^is to 'be a member of a 
club, or some other group which is sponsoring the 
study, and already is familiar with the purposes 
for which the data arc being collected, she may 
respond quite satisfactorily to a telephone inter- 
view. In situations such as this, when good rapport 
previously has been, established, telepjione- inter- 
viewing can be quite effective, and the time and 
anoijey savings becon^e importcmt inducements - to 
decide on this method. In general, though, tele- 
phone interviews are not so desirablefor long, 
tedious vinterviews, regardless of how welT^^Qpport 
has been established. Then, too-, the values deem- 
ing from personal observations are iost, antf it is' 
not so easy to di^b'cover and overcome ikal reasons 
for reluctance in answering. J ' 

Grc'jtj tnterviev/~This techiiique i5 not tfv be 
confused v'ith jrcup counseling or group therapy, 
^.^ertainly no; with group discussion. Essentially, 
it is a device for shortening the data-colIecHng 
phase to a minimum: In this procedure, the inter- 
\'iewcr assembles his prospective respondents ip'o 
a single group or series of groups, and explains the 
purposes of the study to them. 

He gives necessary instruc^ons and explains 
questions which may not be cj^ar. Then the indi- 
vidur .> fill out the form. JPney may be asked to 
answer each question sinraltaneously, with the in- 
terviewer commenting onyeach as they come\to it, 
or rvjch respondent can y^>rk at his own speed and 
turn in his schedule when he has worked thyough 
it to his satisfaction- 

In addition to st^tting u.p the interview and mak- 
ing initial explaiiations IL^ interviewer is there to 
provide whatever help he can to a.ssure complete 
and usable record forms. If ^^c is interested in 



having all of the respondents deal with the 5ame 
question at the same time, he should arrange for 
assistants to circulate among the group to aid those 
who are having difficulty with the meaning of ques- 
tions and to encourage them to answer all ques- 
tions. The interviewer should discourage conver- 
sation which might lead to pooling of answers, 
which means getting one set of answers repeated 
several times rather, than several different sets 
of answers. ' 

The obvious advantage of the method lies in the 
saVings of time and transportation expenses. Then, 
too, the same preliminary explanatory remarks will 
serve for the entire group. Some of the more ob- 
vious trouble 5pots also can be cleared up collec- 
tively. ^ third factor is that there should be little 
or no dilFficuIty connected with gaining admittance, 
establishing rapport>v^ dealing with refusals. 

If the meeting is called for the specific purpose 
of filling out the schedule, or if the members know 
the Interview is to be an item on the agenda, they 
will come knowing the purpose of the meeting and 
b'- prepared to cooperate. Similar reactions may 
be expected from a selected sample which has 
agreed to assemble for purposes of supplying the 
information you need, s 

Limitations of che effectiv^ess of group inter- 
viewing are related, (1) to reluctance of respondents 
to ask about points which. are not clear to them, (2) 
to their overwillingness to skip the difficult (very 
likely the most important) questions, and (3) to the- 
polling problem already mentioned. There is less 
opportunity to probe for reasons for **yes-no*'' or 
**agree-disagrce" answers, so requests for free ret 
sponse or essay type ansv ers are not productive. 

Responsibility of the IntervSewer 

The interviewer has a decidedly important part 
in the successful com^jletion of a piece of research. 
On him falls the responsibility for actually contact- 
ing the respondents and collecting the data. A 
good device for collecting evidence' can be made 
worthless in the hands of an inefficient or irrespon- 
sible interviewer. Because his part is so vital in 
any research project requiring interviewers, his 
selection and training are important steps in the 
research process. 

Before interviewers are selected you should have 
a clear picture of what their duties will be and what 
will be expected of thqm. The infferviewers should 
aiso understand their responsibilities and be will- 
ing to accept them. This knowledge will help you 
rule out obvious misfits, and suitable training 
should make the others able to carry put their 
assjignments in adequate fashion. We can think of 
several personal and professional responsibilities 
of our interviewers. We will discuss each briefly. 
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1. 'Maintain a nedt personal appearance — AsVour 
intorvicwers ropresont both you and the agct^cy 
which is sponsoring you, it'is best that they ser^c 
as good representatives. For this reason. yoi\ 
should try to get pleasant appearing, neat persons 
to serve as interviewers. They stio^ikl-dress appro- 
priately for the season and the environment in 
which they will be working. They should be clean 
and not given to offensive habits or mannerisms. 

2. Locate the respondent, get the facts, and re- 
cord them — The interviewer must be familiar 
enough with the area to which he is assigned to 
locate the proper respondents according to the 

* sampling plan for the study^ or he must learn how 
to use the map of the area. If a person to be inter- 
viewed* is not available, he should report the fact 
to his supervisor. Under ho circumstances should 
he make substitutions without authorization. 

A prospective respondent is not dropped from 
the study or substituted for merely because he is 
not at home at the time of the initial'call. If it is. 
determine 1 that he will be away for the duration 
ot the fiek' work period, that he is ill, or that he 
will not be available for some other reason, it may 
be necessary to list him as ''not available'' or to 
r^ibstitute for him according to the prearranged 
plan. Several calls may be accessary to locate a 
■ person or to complete an interview with him if he 
works, away from home or is busy. 

An interviewer must make every effort to obtain 
adequate and objective answers to all questions. 
Incomplete schedules and partial answers tend to 
"destroy che randomness of a sample and make study 
findings loss useful. The interviewer should have 
a person;j* desire to make every contact fruitful. 
Notations should be legible, and as complete as 
necessary for others to understand. Researchers 
can help interviewers by doing a good job of orient- 
ing them to the study and by providing them with 
good instruments (questionnaires, schedules). 

3. Be considerate and honest with respondents — 

Because yoUr interviewe#*must be able to think on 
his feet and be able to adjust to the situation with 
little or no foreknowledge of what it may be, it is 
desirable that he be an intelligent and understand- 
ing person. He must deal in an honest and forth- 
right manner with all respondents, but he must 
know when to say gracefully that he cannot answer 
a question. However, questions about the purposes 
of the study should be answered frankly. 

4. Understand the purposes of the study — The 

good interviewer realizes that to do his job right 
and to have a personal feeling of satisfaction in 
doing it. he must know what the study is about, 
what its objectives are, and why the particular^ 
questions are being asked. When he understand?*^ 
these points thoroughly, he finds it easier to gain 



an inner conviction that the study is worthwhile, 
and also finds it easier to transmit that feeling to 
his associates and respondents. Similarly, he finds 
that his work becomes more effective and more 
enjoyable. 

' 5. Be thoroughly famUiar with the instruments — 

It is very important that an interviewer study and 
understand very tiioroiighly the forms upon which 
answers are to be recorded. Once he has gained 
personal' understanding, he can transpose the tech- 
, nical language of the project outline into, easily 
understood and meaningful language for his re- 
spondents. ' . 

He can gain the desired degree of familiarity by 
careful attention durmg the training phase.^and by 
asking questions to clear up points which he does 
not understand completely. When he knows the 
*'why's" as well as the **what*s" of his assignment, 
he can do a better job of anticipating questions of 
respondents and of having answers ready for them. 

6. Follow sampling instructions — The interview- 
er must realize that considerable thought is given 
to identifying the people he is to interview, and 
that he has no authority to take liberties with the 

'sampling plan or instructions for its use. .Once he 
is assigned certain persons to contact, it becomes 
his duty to locate those persons and to do all that 
is in his power to secure the information. 

If there are good reasons why he cannot carry 
out the plan, he should report the fact, and explain 
why, so that proper corrective measures may be 
applied. He must realizij that he can make the 
situation worse instead of better by acting on his ' 
own initiative rather than checking with his super- 
visor for further instructions. 

7. Ask questions exactly as written — The inter- 
viewer must realize the difficu*lty which lies in de- 
signing a question which /iieans the same thing to' 
everyone. At the same t'irpe he must realize vi. is 
not wise to modify questions at any time. Even if 
the respondent appears to grasp a different mean- 
ing than the one he believes a questions contains, 
he is not at liberty to interpret the question to get 
what he considers to be* the proper answer. 

Although the interviewer is encouraged to urge 
the respondent to think-and to answer completely, 
he is not authorized to direct the thinking of the 
respondent in any way. He nullifies the purpose 
of the study if he rephrases questions, suggests 
things to consider, asks leading follow-up questions, 
or does anything but encourage the respondent to 
think for himself. 

8. Check work foitrcompletepess J* g) — No 

record ii rm should be turned in by aj. m ^ viewer 
until he has checked it thoroughly foi' cciiipleteness 
arid accuracy. It is most effective when done im- 



mediately after the interview, eithei* befoic leaving 
^•the respondent or before going on to the next one. 
Editing should be very thojough at this stage. 

If the respondent seems to be anxious to have 
the interviewer leave, the interviewer niay thank 
hini warmly for his coopei ation and make his check 
in the car. However, he is in a bettei- position to 
make corrections and completions if lie makes the 
check befoie leaving. Ordinarily, :\ remaik like 
"You have been most cooperative. Do you mind 
if I glance over this form to see that I have eveiy- 
thing?'* will- take the pressure off the respondent 
and make him happy- to let the interviewei' read 
through the form befare leaving. It may even . 
prompt him to volunteer nioje useful information. 

' <.. • ^ ■ i • •• 

Sei**cting[^ncl Training Ijiferviewers 

Extension research usually is conducted under 
the leadership of extension peisonnel. For the 
most pflrt, data collecti-ng is done by extension 
workers and lay loaders.^ Sometimes, experiment ' 
station and other research persons from kmd-gi*iHit 
institutions are involved. It is jareiy the case that 
professional research organizations are employed 
to do research for- Extension. 

Suggestions that follow will, foi' the most pait, 
be "directed toward the untrained interviewer. More 
than likely, these will be county level extension per- 
sonnel pr local leadeis. Extension personnel do 
not receive extra pay for such tasks and local lead- 
ers are ordinarily not paid for their services. 

The ab5;ence of direct economic motivation sug- 
gests tv/o possible limitations on the effectiveness 
of interviewing by such persons. For one thing, 
strict attendance and punctuality cannot be de- 
inanded of interviewers and, for another, there 
may be less feeling of responsibility on the pait of. 
volunteers. However, we in Extension have been 
very successful on many occasions in the use of 
volunteer lay leaders and county extension person- 
nel in our research teams. 

Interviewing is best done by 'people who are in- 
terested in doing the job, and are willing to main- 
tain an objective approach. Before agix»eing to 
help With a study, anyone offering his services .as , 
an interviewer should feel reasonably certain that 
he will, have enough free lime to stay with the job 
until it is completed. 

You should mention the probable dui ation of the 
work when you make inquiries abruit interviewers. 
You should includ^ time for training and other 
orientation activities, as well as time for doing the 
interviews. You should allow no one to participate 
in data collection for the study who does not par- 
ticipate in the training phase, and wlio is not will- 
ing to assume all pioper responsibilities of inter- 
viewers. 



Foi* most studies, you can work lay leaders into 
your intei viewing plans with little difficulty. You 
have no housing problem with them and the trans- 
poitation pioblem is minimized. They probably 
know the aiea quite well and know how to get 
around in it. 

Thqy are more likely to have a personal interest 
in the outcome of the *^udy and may be in a posi- 
tion to capitalize" on the findings. If properly 
cautioned, they can be counted' on to get accurate 
information, /rheir familiarity with the local scene 
can be used to advantage in identifying and locat- 
ing respondents. ^ 

Regardless oX experience, all interviewers re- 
quire a certain aiiiount of training foi* a specific 
I'csearch - undertaking. * The inexperienced inter- 
viewer will require more tracing in the general 
philosophy and discipline of interviewing, but all 
of them will require orientation for the particular 
research problcpi, and experience in dealing with 
the specific rcsearch^instruments which will be 
used. ? ' ' / . • 

We recommend that all interviewers be given 
training in six areas, with the intensity'of the train- 
ing in (>ach area to depend on the experience Ipvel 
of the interviewers for the particular study. The 
suggested areas are: (1) Background and purposes 
of the study, (2) sampling .procedur<?s and how to 
apply them, (3) interviewing procedures and tech- 
niques to be used, (4) familiarization with the re- 
search instruments to be used, (5) experience in 
filling in and editing thci record form, anu (6) 
experience in asking the questions. An actual 
field test to follow the tra -j;ing session isi very 
effective in making the transition from theory to 
reality. " 

Almost invariably the length of time that can be 
deyoted to training'the research team is lin'kited. 
^'This necessitates careful planning and efficient use 
of linie. - A minimum of six hours is recommended 
for larger studies, although a longer period, is de- 
sirablc: * 

It is a good idea to divide the tiainingMiito two*^ 
parts: (1) Orientation (purposes, ba.ckground and 
techniques); andM2) practice (experience in filling 
in and editing the record forms and in asking the 
questions). We have used the following outline on 
several occasions in a onp-day training session for 
some quite' substantial studies. It requires at least 
six hours but might lake more, depending on the 
amount of discussion and elaboration you permit. 

Agenda for Training Interviewers 

1. Why we are here. (Resident agent in charge.) 

a. Greetings 'iuid introductions. 
' b. Purpose of meeting. 
- c. Explanation of plans for the day. 



5. 



6. 
7. 



8. 

1 

9.'^ 



d. Why we wanted to make this study; how 
wit} plan to do it. {1> Review of objectives. 
What an interviewer docs. (Trainuig Jeader 
takes over.) ^ ■ r 

a. Importance of the interviewer in y research 
team. ( 1 ) In^'portancc to success of i)raject. 

b. Definition of term.s. (Respo;ident, sample, 
schedule, refdJjal. call-back, unit, and other ; 
jjeneral terms. Al.so any special terms used 
in the pai:ticular study.) 

c. The sampling plan: what it is and how to 
follow it. 

d. ^ Detailed explanation of instruments. (Qucs-^^ 

tion by question and with opportunity for. ^ 
. questions from intei vicwers for clarifica- 
tion.) 

e. General - instructions for "intei* viewing. 
(Have a set of instructions to distribute to 

^ interviewers but discuss them in detaiLat 
this point. Ask for questions on any point 
• which noi clear to alt) 

f. Specific instructions for asking questions. 
^(Alter a general sta^tement regarding tech- 
niques "bf asking questions, use the ques- ' 
tions in the study instrument to demon- 
strate how to ask them properly. Also 
demonstrate suitable forms of approaching 
fhe respondent and. of terminating the 
ihtervieW.) - , 

Filling in and editing schedules. (Have inter- 
view:crs pair off and take turns asking the 
questions, and answering .theiji. Each then 
edits the schedule he has taken.) 
Editing by trainers. (Ti'tiiners check sched- 
ules turned in during prac|tice and point out 
errors to Xhe group to alert them for similar 
situations in their own interviews. Trainers 
explain how to avoid errors and how to go 
about correcting them.) 

Explaining special procedures for the study, 

a. Procedures for identifying respondent, 
making contact, arranging for interview; 

/ dealing with refusals, not-at-homes, call- 
' backs. 

b. Where and when to bring completed sched- 
ules. 

c jlow to contact leader of survey party for 
further instructions or help in dealing with 
unexpected or unusual situations. 
Making first assignments for interviewers. 
Setting time for general meeting at end of 
V 'A day of interviewing to discuss problems, 
clear ijp questions, and check on quality of 
work, 

Sujiiary of day's activities; final request for 
questions. 

Closing words of encouragement, assurance 
ajid morale' boosting. 



Inierviev/ing Procedures and Techniques \ 

There are four main stages to making and conv 
"t)lcting*an interview. 

1. Gaining entrance, making the approach and 
establishing rapport, 

2. Securing and recording information. 

3. Closing the interview. ^. 

4. Editing." 

) 

Esfablishing Rapporf — A good interview depends 
to a great extent on a quickly established feeling of 
trust and confidence in the interviewer by the re- 
, sp'ondent. Sincere friendliness on the part of the 
interviewer is a firm step in the direction of 
achieving this objective. Techniques for gaining 
admittance to a respondent's home and for estab- 
lishing friendly relxitions (rapport) vary with the 
situjttion. A good interviewer will learn how to 
make on-thq-spot adaptations to fit most situations 
likely to confrpnt him. 

Some techniques may be acquired from study of 
literature on the subject, but practice is also im- 
portant. An interviewer needs to try to under- 
stand the reasons for^- resistance and to think of 
ways to overcome them. The niost common rea- 
sons lor resistance are shyness, fear of being sold 
something, feeling of inability to answer questions, 
and reluctance to talk about personal matters. 

Neat personal appearance and appropriate dress 
are im.portant in creating good first impressions. 
Equally important is the general tone of your intro- 
ductory remarks. They should (1) identify you as 
the interviewer, (2) explain briefJy the reason for 
your call, and (3) ask for the respondent's cooper- 
ation. 

One interviewer has used successfully the follow- 
ing as an opening sentence: '*Good morning (good 
afternoon, hello). I am (name).'' Pause. **May I 
c6me in and visit with you for a few minutes?" This 
provides an opportunity for the respondent to give 
his or her name, if it is not already known to the 
intervieVvcr. 1^ 

Most people invite the interviewer in on this re- 
mark. However, if there is hesitancy or if some 
question is asked as to the reason why, the next step 
is to explain in a little more detail what you are 
ddihfi and why. It may help to include sorne state- 
ment\p let the respondent know that cooperation 
is important. **Ypu are one of a few people select- 
ed and your opinion (cooperation) is very impor- 
tant," is helpful. 

''How was I (fny name) selected?" is a question 
often asked by a person being interviewed. The 
answer^ to this should be' honest and brief. Some- 
thing like, **Well, we u.'t visit everyone, so we are 
talking to every fifth person on your block (in this 
neighborhood, on oiir list, and so on)," usually will 
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satisfy the respondent. Sometimes assurance is 
needed that names will not be used in the report, 
or that the. interviewer is not selling a product. 

An alert interviewer usually can pick up some- 
thing of common interest to talk about if a person 
shows signs of wanting tb cut off the .conversation. 
It a homemaker says she is busy, you can recognize 
the fact, express understanding, and malce an ap- 
pointment for a return call at a more convenient 
time. 

Refusals should not be accepted except as a last 
resort. If an interviewer "has a high number of 
refusals, he may need some help with his tech- 
niques. On the ether hand, it' might be in the best 
interests of the study to relieve him of his inter- 
. viewing assignments. Sometimes, the person who$e 
sensitive nature makes him a poor interviewer can 
make a valuable contribution to the study as a 
discriminating checker or, later, as a clerical aid 
in tabulating the data. 

Almost anyone can be interviewed if you use the 
right approach. Do not force the interview on a 
person, and never become impatient or angry .7^ If 
he refuses, try to make an appointment for some 
other time. If this also fails, thank him for con- 
sidering your request and leave, showing no resent- 
ment. This leaves the way clear for another inter- 
viewer — the crew leader, an older* (or younger) 
interviewer, or one of the opposite sex — to make a 
try at gettihg the* interview. 

Quite often, the respondent will give .an inter- 
view when another interviewer approaches him — ' 
possibly because he now realizes the importance of 
the study or, maybe, because he regretted his orig- 
inal refusal but didn't want to back down. 

Refusals happen^ very seldom so do not anticipate 
them. Try to avoid them but, if you can't, accept 
them graciously. Then forget it. Remember,, there 
will be some refusals in any survey, so don't take it 
personally if you get one. 

Before giving up on assignments, make at least 
two call-backs where the respondent was not at 
home on your first call, or was too busy to be inter- 
viewed. Be sure to keep a record of incomplete 
interviews, explaining what you did, and why you 
had to give up on them. 

Securing Information — Thorough familiarity 
with questions helps to make a smooth interview. 
Under no circumstances roay interviewers change 
the wording of questions. However, you can re- 
read them and define or explain a worcT if you feel 
the respondent does not understand it. 

Explain that you are going to write down replies, 
and why. If you read back answers to free response 
questions you assure respondents that they are not 
being misquoted. This sometimes helps to secure 
more complete replies as respondents may add to 
their original remarks if they desire. 
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Do not be afraid to wait for, a reply. Time may 
be needed to ^orni thoughts and to think of ways to 
express thent? A smile, or ^That's all right. Take 
your time in answering," sometimes helps. Be a 
good listener. Let the respondent talk. Do not 
rush him but try to discourage talk about inciden- 
tals if they have no relation to the study. 

Be careful to follow questions in the order of 
their appearance On the schedule and neivei* omit 
any that apply to the -person being interviewed. 
However, be alert to notice when a person answers 
a question during conversation that is to be asked 
.later. 

Replies must be encouraged without appearing 
to agree or disagree. Agreement may influence a 
person to give future answers in terms of what he 
or she thinks you want. Disagreement may cause 
a person to suppress real opinions. Watch for re- 
actions from the respondents; make marginal notes 
about them. 

If interest appears to lag, some statement as to 
the time required to complete the interview may 
help. A change in tone of voice, or a shift in pace 
may help. Transition statements from one area 
of a schedule to another also help to maintain in- 
terest. For example, if you have been asking ques- 
tions about clothing, you might say, **You have 
answered the questions on clothing. Now I have 
a few others about' foods.'* Or, l*The \nformation 
you have given us about crops raised last year will 
be very helpful. . Our next questions are about live- ■ 
stock/' ■ 

In opinion or attitude questions, 'it is very impor- 
tant to assure respondents that there are no right 
or wrong answers. Frequently a person will ask 
. an interviewer, **Is that what you wanted me to 
say?'* In. this case, you can say, **Yes, if that is 
your opinion. There is no one answer to this ques- 
tion.'' 

More skill and patience are required to obtain 
replies to free response (open-end) questions than 
to checkoff questions, A friendly tone of voice 
and a conversational manner will encourage a re- 
ply. **I don't know," or '*! have no opinion/' are 
easy ways to avoid expressing real feelings. An 
interviewer must find ways of getting past such 
replies to get usable answers. Sometimes repeat- 
ing the question in a different tone Of voice helps. 
Sometimes careful probing is justified. 

If probing is used, you must be careful not to 
use suggestive probes. For example, you might 
ask the question, **Why do you like a 15-minute 
radio program on one subject?'' A suggestive 
probe would be, **More interesting?" This sug- 
gests a reply, ami may be pounced upon by the 
respondent regardless of what his real reason 
might have been. 
Once (he person has started to talk, the inter- 
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viewer may probe for more complete answers by 
comments- such as, '^Anything else?'' *'Other ideas?'' 
^'Another comment?'' or 'V\re you sure that is all?" 
These do not suggest answers but do encourage the 
respondent to con;e up with a more complete 
answer. ' 

It should be noted here that there are occasions 
when **n6 0Rini{)n" and '*don't know" replies are 
valid answers. 'If a person really has no opinion 
or does not know, he should not be forced ittto 
replying further. More harm than good may be 
dcfne by trying to force an answer if the respondent 
is reluctant to reply to questions. Rather than 
antagonize the respondent, the interviewer simply 
writes ''refusal" or '*did not choose to answer" be- 
side the question. 

A final reminder while securing informati^: 
Write clearly and make all entries as complete as 
possible. Although they may seexn perfectly ob« 
vious when recorded, sketchy notes, unfinished 
phrases, or single words often prove to be meaning- 
less or ambiguous to the person who must tabulate 
the answers. While reading back th'e reply to the 
respondent as a check on accuracy and complete- 
ness, try to imagine what a person, who must react 
to the recorded entry only, will get out of it. 

Closing the Interview — A pleasant impression of..^ 
both study and study personnel should be left with 
the respondent. He or she should be left with a 
feeling of having been helpful and that the coopera- 
. tion has been appreciated. Always extend some 
expression of appreciation of the close of the inter- 
view. 

Try to have the parting be such that your return, 
either as an individual or as a member of this or 
another research team, would be welcomed. You 
Viever know when you may have to call back about 
some item, or follow through on some afterthought 
related to the study. !t is a matter of good common 
sense and professional ethics to try to preserve 
friendly relations for future studies, whether by 
you or by others^ which may involve your re- 
spondent. 

Editing — Too much emphasis cannot be given to 
the importance of checking the completeness, accur- 
acy, legibility and consistency of each schedule be- 
fore you turn it in. Careless editing by the inter- 
viewer can bias the results of the entire study. 
Never fail to go over the schedule, either before 
you leave the respondent or before you move on 
for the next interview. In this way you can clear 
up discrepancies and correct them. 

Be sure all identifying information is entered. 
Write out abbreviations, complete fragmentary 
sentences, rewrite hard to read passages, and ex- 
pand sketchy remarks. AJake marginal comments 



to explain apparently inconsistent or unusual 
entries. Sign the form so that others will know who 
to contact for further clarification or follow-up 
tasks. 

Although your work will be edited again, both 
by you and by others, the editing ybu- do at this 
point is* likely to be the most effective in improving 
. the overall quality of the study. 

Interviewing Without a Schedule (questionnaire 
or record form) — Most extension studies are made 
with the aid of some kind of record form. ..When 
interviewers are involved, sonrte kind of schedule 
almost always is used. However, th^s^ne are situa- 
tions where it is not convenient or advisable to use 
a standardized record form. Oftentimes, in explor- 
atory research particularly, a carefully structured 
questionnaire or schedule will hinder more' than it 
will help. This type 'of interviewing will not be 
discussed to any extent in this chapter. 

"Unstructured," "focused," "depth," and "non- 
directive" are terms applied to interviews made 
without apparent aid of a record form, The inter- 
viewer, sometimes called the "operator" in this 
type of interviewing as he may work with a tape 
recorder, dictation machine or similar device, ap- 
pears to carry crti aimless conversation with the 
respondent. He or an assistant records everything 
that is said . \d analyzes the comments after, they 
have been typed. 

Actually, the interviQwer-operator directs the 
conversations by carefully inserted comments of 
his own so that all the areas he desires to have men- 
tioned or discussed get worked into the conversa- 
tions. He may work with a memorized que sheet 
or he may have a topic outline before him while . 
talking to the respondent. ^ , 

Although more involved, and difficult to trans- 
cribe, these same techniques may be used for group 
interviews. No doubt, we have all seen them 
demonstrated in radio or television interviews when 
at least the announcer knows what information he ' 
wants to come out during the interview. 

Although these types of interviews have their 
place, ana are increasingly important in extension 
•research, they have to be ruled out for- much of our 
research because more skill and training is required'4^ 
of the interviewers. Spedal probing techniques 
must be mastered to get results without appearing 
to direct the interview. 

Data are^ more difficult to analyze' in this form, 
from both a time and a skill-of-operator standpoint. 
They do not contribute so readily to statistical 
analyses and, as a result, are limited to preliminary 
or exploratory research rather than research to 
servQ as a basis for generalizing. 

We are becoming increasingly aware of a need 
in extension to go beyond the usual descriptive 



type of study we do so often, and work out ways 
of really getting at what people think, what they 
want, why they change, and how they are motivat- 
ed. These *'nondirective" techniques may be ones 
we will have to adapt to our needs if we would keep 
up with the demands of our work. It is hoped that 
more extension researchers will desire to become 
better trained in these techniques. At the moment 
they are relatively nfew, but there is some good 
literature avail, ble about them for those who are 
interested. If an extension worker is interested in 
the type of interviewing and doesn't have the need- 
ed training, other help should be sought. 

Summary 

For extension research purposes, we think of 
interviewing as the more personalized ways of 
securing information, the face-to-face or person-to- 
person types of data gathering. We also think of 
the kind of interviewing in which a schrJule or 
questionnaire is used rather than wher th: more 



elaborate and demanding nondirective techniques 
are employed. 

We believe it is important for a person directing 
research to be aware of what interviewer responsi- 
bilities are and we discussed some of these respon- 
sibilities in part. Next we mention some things to 
consider in training interviewers and include a sug- 
gested outline for a six-hour training program for 
interviewers. 

Then we elaborate on the four steps of the inter- 
viewing task — making the contact, collecting the 
information, closing the interview, and editing the 
interview form. We close the chapter with a brief 
discussioi,! of interviewer 'techniques for situations 
where routine interviewing methods are not effec-* 
tive. We suggest some of the reasons why the new- 
er techniques, for the present at least, are not 
readily adaptable to "extension research and -evalu- 
ation, but we go on to state why this is the direction 
in which extension researchers must .explore to 
find ways of dealing with the growing complexity 
of extension problems. 
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-Chapter Xllt ^ 
DEVICES FOR COLLECTING DATA 

Gladys Galhfp • 



A good deal of behavior can be observed through 
the senses. recognr/e improvement in terms of 
accepted standai ds in a 4-H Club boy or girbor an 
adult leader by watching the person pi*eside or by 
hearing him talk at a meeting at different times. 

We can drive through the country and observe 
the increase in compliance with soil enrichment 
practices, the use of suggested varieties of grain, 
the improvement in livestock, and the betterment 
in farm and home buildings and yards. We look 
for evidence of favorable change in light of whaf 
ue have been leaching. Even changes in intangi- 
bles such as attitudes,' interests, and personality 
traits are expressed in ovdrt behavior. Such obser- 
vation is good, but it is nql sufficient. 

The Need for Measuring Devices 

Different kinds"^ of devices have been designed 
to measure individual behavior. These are devices 
which for the most part gathjer .evidence as to it- 
person's knowledge, attitudes, oi'^His activities, put 
in another way, what a person knows, feels, or does. 
These devices take but a few minutes to apply, 
whereas direct observation of behavior by compe- 
tent observers often requires many hours or^days 
and is influenced by observer experiences and 
understandings. 

The questionnaire or other device used to eval- 
uate a program should be Worked out especially 
for that particular program. We cannot usually 
expect to find one ready made because differences 
in objectives or in the method^ of approach gener- 
ally make a technique that is effective for one pro- 
gram unsatisfactory for another. However, before 
(constructing a new device, it is helpful to examine 
some of those which have been used previously. 
They may be a sotiri^e of suggestions for the par- 
ticular problem upon which we are ^vorking and 
may furnish test patterns for construction. 

The Selection of the^Measurtng Device 

When selecting the kind of devices to be used-, 
we should Tceep two things in mind: (1) The teaching 
objective, and (2) the evidence for*which we need 
to look in order to learn what progress has been 
made in reaching the objecfive; 



In the educational field there are numerous de- 
vices that can b'e used to measure progress toward 
an educational obje^:tive. Some of those which are 
designed to observe and record behavior are: 

Knowledge Questions— It is important to know 
whether a person knows infonnation and the level 
• of information. ^ 

Understanding Questions — It is important to 
' know whether a person understands or can apply 
certain acquired knowledge of principles. 

Skill or Performance Ratings — These are used to 
determine skill being performed or the product 
resulting. They may show the amount of skill the 
homemaker has or to what degree skills have been 
acquii'ed, ' - . • • 

c 

'Attitude Scales— These are used to shc^w how 
people feel toward things, whether they are for or 
against certain questions, social issues, or problems. 

Interest Checks — These are used to find out in 
what people are interested. 

Confidence Testing — A measure of self-assur- 
a^nce, the. feeling that you can do a certain thing. 

Adoption of Practices — The adoption of a recom- 
mended practice, is frequently used , in extension 
reseai ch as evidence that the farmer or homemaker 
has changed behavior and acquired new knowledge, 
skill, and understanding. 

Some Illustrations of Devices 

Knowledge Questions or Checks on Levels of 
information — Obviously it is essential to impart 
many facts in order to teach any subject matter 
in Extension. Whe^n we test we can ascertain 
rather easily. whether the persons taiight' acquired 
this knowledge. But can be assume ih^t they 
understand -Wiose facts which they have learned? 
Certuinly we^should not assume that they are able 
to -apply their knowledge. If our objective is to 
develop understanding it is necessary to extend 
our questions in order to learn if this knowledge is 
comprehended*and applied. 

1 ■ / •. 



In the schedule used in a Wisconsin Study- are 
some illustrations of questions which atf?mpt to 
get at knowledge only. 

1. Some fertilizers are higher in plant food than 
others. Which one of theseiertilizers has the 
most nitrogen? (Check one only) 

( ') a. Anhydrous ammonia 
(- ) b. Ammonium nitrate 
f ) c. 0-20-20 
( ) d. 10-10-10 

2. The acidity or sourness of a soil is sometimes 
expressed as a pH number. Which soil is the 
most acid? (Check one only) ^\ 

( ) a. pH5 
{ ) b. pH9 

3- Acid soils have a number of bad effects. Which 
one of these things does an acid soil do? 
(Check dne unly) 

( ) a. Ties up potash so crop can't get it 
f ) b. Ties up phosphorous so crop can't get 
it 

( ) c. Makes soil hard. ^ , 

( ) d. Keeps soil from holding water 

4. How many pounds of nitrogen would there be 
in 100 pounds of 3-12-12 fertilizer? (Check 
one only) 

( ) a. None 

( ) b. 3 pounds 

( ) c. 12 pounds 

{ ) d. 27 pounds 

5. Some crops have ''hunger signs'* which show 
that they are in need of plant food. WhQfi 
alfalfa leaves have white dots around thk 
margins and the lower, leaves dry up, whst 
plant food are they probably short on? (Check 
one only) s* 

( ) a. Nitrogen . \ 

( ' ) b. Phosphorous 
( ) c. Potash 
( ) d. Calcium 

6. You've probably lieard or read that you 



shouldn't plant earn on a 9^h slope.^ How 
steep is 9%? (Check one only) 
( ) a. 1 foot drop in 9 feet 
( ) b. 9 feet drop in 1 rod 
( ) c. 9 feet drop in 100 feet 
( ) d. None of these 

An example of a knowledge question in the field- 
of marketing information for consumers was taken 
from the Lake Charles, Louisiana Study.' This 
question was designed to get at the l.cvel of infer-' 
mation of homemakers in Lake Charles. 13, 

7. If you were going to buy a higher grade cut 
of beef, what is the main thing you would look 
for? (Check one only) 



( J a. Leanness, or free from fat 

( ) b. Presence of a yellow rim of fat 

( J c. Deep red in color 

( ) d. Little lines of fat running through the 

meat * 
( ) e. Don't know • • ' ^ 

This question was asked-as an open end question. 
The enumerator tll^n checked the corresponding 
answer on the schedule. 

Other examples of knowledge questions are taken 
from the Arkansas Extension Pilot Cotton Project'^ 
on disease control. * 

8. Angular leaf spot is carried primarily from 
' season to season by which'of these means? 

(Check one or more that apply) 

( ) a. On the. seed 
( ) \. In the soil 
( ) c. Plant refuse 

( J d. Mechanical equipment ^ . 

{ ) e. Don't know 

Below is a series of questions about common cot-* 
ton insects.-' The interyiewer gave the fespoiKient, 
a picture of a common cotton iiiSect and asked ques.- 
tions about it^ The interviewer then checked or 
wrote in the replies on his record form. ^ ' 

9. What is the name of this insect? 

( ) a. Bollworm * 

b. Other ' -.^^lI , 

( ) c. Don't know 

10. A.t what stage of growth of the cotton plant is 
it recommended that farmers start spraying 
or dusting to control this insect? 

( ) a. Pre-square 

b. Other J : 

( ) c. Don't know 

11. What is the standard or indicator for deter- 
mining when x:ontrol measi" Gs for this insect 
should be applied? 

( ) \^ 10 to 25% punctured square — dry 
' year 

( ) b. 25 to 40% punctured square — normal 
year 

c. Other : 

) d. Donlt'-I^now 



( 

12. What kind of insecticide is effective in con- 
' r trolling this, insect? 

a. : L_,.. . [ ■ 

( ) b. Don't know 

How often should this insecticide be applied 
to effect continued control of this .insect? 

a. 1— 

( ) b. Don't know 
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14. I low much of this insecticide should be ap- 
plied per acre, niid.season; how jiuJch late 
season? 

Midseaso:} Late Season 
Dust a. - - (1. 

Spray b. e. 

DoiVi know c. - 1. . • 

15. When do you cjuil applying insecticide for con- 
trolling this insect? 

( ) b. Don't know 

Understanding Questions — It is important lo find 
out whether a p(MT.on understands or can apply cei - 
tain acquired kn A'ledge. 

Comprehension or understanding usually can be 
inferred from what the respondent says or docs, 
Present him with a new situation and see whether 
he can apply the basic principles taught him. The 
.situation must be natural and not so new but that 
he can see the similarity to former situations, 

There are various levels of understanding and 
degrees of understanding. Trying to* measure at 
too high a levef or too ideal a level will be dis- 
appointing. On the other iiand» to measure at too 
simj)lo or too low a level will not tell you whetlier 
tile person can apply the knowledge at his own 
level of living and working. 

Here are some illustrations of niethod.s' u.scd to 
lest understanding:- 

1, Teaching selection of accessories for the basic 
dress or suit.*' The home demonstration 
agent or a local leader wears a basic droj ; or 
a basic suit and teaches the selection of acces- 
sories to be worn with it. 

After the teaching is finished a second 
assortment of accessories, including hats, 
shoes, scarfs, jewelry." purse, and the like, are 
brought before the group. The members in 
attendance are asked to select appropriate 
accessories for tiie agent or leader to wear 
for two or threr uiiforent occa.^ions. This will 
indicate if they know the principles and if 
they can apply them. 

2. In teaching farmstead planning** in workshops 
to those farmers and homemakers interested 
in building, there are certain principles taught 
as to location of buildings on the farjiistead. 

The extensioai vsorker asks each husband 
and wife present to draw a rough plan of their 
farmstead, showing slope of ground, direction 
of wind. view, location of high^vay and drive- 
way. Then, using circles to indicate build- 
mgs. they put in the house and the farm build- 
ings, applying the principles of farmstead 
l>lanninR, This tests their knowledge of rprin 
ciples and the ability of the husband and wdV 
to apply principle.^ to their own situation. 



3. In teaching ar t principles' and the application 
of art principles in design and color in the 
home at the local leader training meeting, a 
mimeographed sketch of a living r (Mjm includ- 
ing doors and windows is given each leader*. 
Each person at the meeting is asked to ar- 
range the furniture according to principles 
taught. Then swat hes^)f niatt^rials and wall- 
paper are used to work out the color scheme 
for the room. 

4. Problem-solving 'devices'* can be used to meas- 
ure understanding." The following is an cx- 
ample of the type of questions that can be 
used by 4-H Club boys and girls: 

A farmer wishes to market liis spring crop 
of hogs before the usual drop in market price. 
Which of the following practices will help? 

(Check one or more) 

( j 1. Have pigs farrowed iri March or early 
April. 

( ) 2. Have pigs farrowed late in May. 

( ). 3. Put brood sows and litters on pasture 

with self-feeders of shelled co^rn and 

tankage in separate compartments/ 
i ) 4. Wean the pigs at 6 weeks of age, 
( ) 5. Provide self-feeders after weaning, 
f ) 6. .After weaning, put pigs in dry lot and 

feed' entirely on corn. 
{. ) Limit the feed during the summer and 

have hogs on fall crops of corn, 

Rciisoris: {Check the statements below which 
i « i-'-v/nt your reasons for choosing the prac- 
ilci: . \nu checked in the preceding list.) 

{ ) a. The usual market drop which affects 
the spring crop of hogs starts in Sep- 
tember. 

( ) b. The usual market drop which affects 
the spring crop of hogs starts in No- 
vember. 

( ) c. Under favorable conditions hogs will 
be ready for market at 6 months of 
age or less. 

.( ) d. Hogs gain most rapidly when hand fed. 

( ) e. Hogs can usually be produced most 
cheaply on a limited grain ration for 
the^Mifst few months, followed by 
heaVy ^beding. 

( v) f. Brooiirsows and their pigs are able to 
'balarite their own ration \Vhen provid- 
ed "fVee choice'' of tankage and corn. 

Skill or Performance Ratings — There are two rea- 
sons for nieasuring\krlls: U) 'IV' find out if the peo- 
ple have acquired a warlicular skill that has been 
*aught and (2) to (iiw out how many skills have_ 
been attained. 

Skills ^^vnt are important for the clientele to learn 
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are listofl. critiTia ;ir4« s(^t for iiumsui iiig the skill.s, 
aiui some (Irvict/s are picpaied iov use. 

L'suall\ .si()r(» cards aiul ralini; scales are us(vl 
to measure skill, perloi niance. and ability. They 
are used extensively in Extension at fairs apd other 
events in judging products and in setting standards 
of (juality. 'I'he product or skill to be rated is care- 
fully analy/od. each aspect is considered separately. 

It is necessary not to have too many points on a 
score card or rating device and theie should be a 
description of two or niore levels of quahty for eacli 
ect of the product, skill, or ability to be rated, 

i'he examples below are devices for judging 
selected skills in a 4-1 1 Club clothing project. With 
each skill is explained exactly what the Mater'' 
should look foi* in rating that parlicuhir skill, 

1. Score Card for Judging Garments 

(Adapted from device constructed by Lois Co; 
bctt. "Michigan Extension Service, tor judging 
4-1 1 Club clothing p» ojects.) 



Club Member 
Local Leader. 



Rater 



Age 



Directions: Hate garinent on each iten which 
apptie^^to it. rating 1. 3, oi* 2, dependiiig upon- 
wlielher the quality cori;isponds to the de- 
scrij)tion in the leit hand column, oi* the right 
hand column, or falls between the two. Add 
scores at bottom ol" page. 

Two or thi-e(» items are shown for ''constrvc- 
tion of a dress," 



STATE- 
MENT No. . 



SCORE 



sijtj in 



rr-..-v..i. 

Ir. fn 



\.t\ . I> . 
I., ?l .1.1 ~ 



2. Judging Clothing Skills 

These are the instructions to l!ie local MI 
Club leadei : "This chart is prepared for use 
in judging the sewing skills tht? girls are 
developing in their 4-11 clothing woik " 

'The sewing skills of each girl are to be 
Judged twice, once at the beginning of the 
project (or as soon as you have an opportunity 
to t^bserve her skill and make a judgtuenti and 
a second time towartl the end of the project. 

"Make your best judgment of each skill fev 
each girl using tiio following yardstick: 

Yardstick for Judging ( lothing Sr.; ' 
(Five Degrees of Skill) 



(1) Practically no skill — does not f* /: 
rJ.) Little skill — needs much helj' *vii.:i: 



ili) Fair skill — but still nced.s some .hc^.p a. id 
practice. [ 

i4l Good skill — needs very l.ttlejit^lp or prac- 
tice. 

(5) Excellent.skill — does lot need any-iielp or 
practice, 

"In the exami)le below a-.d on the folio n;; 
j)ages the numbers in the squai ^>s lof''*' " ih<^ 
numbers on the yardstick. > ;cle 

around the number which slvr^b y. judg- 
ment of the girl's skilL The top row oi the 
pair is for the beginning of the jprojec* i:Vy\ 
the lower row is for thd end. The deference 
between the two ratings shows l.ow mucn the 
girl has developed that skill. 



Example: Skill in 
threading needles. 
(Consider how 
thread and needle 
are hold and speerl 
in threading 
needle.) 



Be«;inriinj4 


9 


2 


3 


4 


5 




1 


2 




4 


5 



-«ctK 



"This, means that at the beginning of the 
project this girl had '^practically no, skill" — 
she didn't know how to thread a needle. But 
during the clothing work she improved until 
at the end she had developed *Tair skilf 
but still needed some practice and help, 

"Some girls in some skills may have Tair 
skill" (3 on the yardstick) to begin with .rn] 
go as high as "excellent skill" (5 on the yard- 
stick) by the end. Others may make no inv 
provement at all. Still others may begin at 
"practically no skill" and end at ''excellent 
skill.'' O^hor^s may begin the work this year 
with ''excellent skill" and of course end with 
Vv .vMent skill " ; 

•/.-c sure to base your judgmeni each time 
:m youi" observations of the girl doing the 
skill and on the quality of work she does. 
Since some girls may not happci: lo perform 
onie of the skills, you will not be ible to 
jiKlge every girl on every skill. Use a .separ- 
ate pamphlet chart for each gi»] " 

3.^Local Leadership Rating Sc:3Ee5 

Extension workers have used different types 
of lating scales with local leaders A pftiftt 
rating scale is sometimes used which indicates 
a scale of specific needs: 1 to 5 standing for 
very much, much, consideraole. some, IHtle 
or none, respectively. 

Por example: How' much help does tl is 
leader need in conducting a discu.ssioii ineet-- 
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Vrr> 
much 



Much 



I 'on sir l- 
crablo 



Soriu* 



Littlf or 
none 



Amaunl of IU I\) Lueai Leadi^r >*( ( (l.s 



be 
in 



Atfitude Testing — The cxtcMision worker \< con- 
cerned jibout the attilude.s [uul beliefs of people 
Sometimes people iiidieiite a disposiUon for or 
.^aiLjainsl homethini;. or sometimes they indicate 
\nuertainty. A person may be ./vorabiy incHned 
toward hi^jh riyid })riee supports ^.e niay fnvor 
flexible price supports, or he may be in do\dJt a-^ 
to which he prefers. Attitudes and beUefs car. 
identified only when |)cople are in a situa^aon 
which they are confident that they aie fr:K» 
exj>resss them.seh es as to vvliat thcV feel or tli 
•'State Kxtension Workers' .nallots" illustrate 
aftiluile device employini; rrjultrple choice (p«e.s- 
i.tons. Ihv extension worker can express hov hu 
feels on a number of questions. There may be 
several (juostions. The. first <|uestion is niore'geii- 
cra!. Ihe last question nioro specific. 

The ballot .cjivcii a person an o|)portunity to ex- 
press his views on a nun^ber (^f (|i'7stions related to 
the rCxtcMision Service Directioiis read as folIiWs: 
'T;<dou each question yon will fine five possiWu, 
ans\\\v'rs Kead all live answers, ami choose tiie 
one that conies nearest to ixpressing the way you 
fee! :d)out the question. Tlien vote by putting? .t'»n 
**X'* at the I.'tt of the answei* you have chosen. ' 

"y'nlc for only one answer to each cjuestiori. If 
noMc of the five answej-s to a (pjcstion happ ris lo 
agree exactly with your own opinion, vole tor the 
one lh:t' comes closest to .-avirn^ :\\luI you think 
iibout it If you want W, write any comments on 
the balk)' . that will id! ri|;ht; bur be sure to vote 
on one questior t)y ui.ikint^ an X. " Use the back 
of the pages for y(jur connnents if you need more 
room to V. rite. Vou do not need to put vour nan^e 
on the lirllot/' 

1. How ({o you feid about the Kxtension S^^i vice 
a:> an institution 

i ; a. There are nuiny se: .ou.. wcakne.^^s^rs 

\v, the Kx en ien Service. 
* ) b The Kxtension Service has ,sn?Mc weak- 

ii( :s(*s. bid none are Uh) scri'^us. 
: I c. The Kxt^ nsii;.i Service h''S • bout as 

:!iany qood points as bad om, . 
? i d The Kxtension Scrvicr: has inoi * r'ood 

p(>:nts ihan bad points. 
( ) e. The Kxtension Seivi " has no serious 

weaknesses.. 

2. How nnich halp (U you ^'et ' •oni your District 
Siq)ervisar in doin^ your ..nrk'' 
< ! a. He never helps at all in uiy v/ork. 
i J b. He hanlly ever helps. 



. ; J c. Js sometimes fairly hc^o' ' 

f id. Is usually (juite ^ ^Ipfui- I 
( » e: Always {^dve.s all - 'le ^iolp could 
possibly want. 

li, L\ MY OPI.XIOX, separate 4-H Cliib subject- 
matter leaflets and bulletins need to be pre- 
pared for u; rj in 4-H Clubs, rather than using 
materials prepared for adults: ,^ 
Yes 

Yes, depending upon age 

of members (2. 
'^/es, depending upon, c^u^j ; i: 

matter ♦ ontent {3i 

■ - (4) 

No -opinion (5). . . 

4. Xow we would like to have your opinion on 
some ideas. I shall I'oad eight statements to 
you and would W^f^ lor you to tell me whether 
you agree or disagree with the statements. 
You may not have an opinion oi^ some of them. 



AGREE 


j aiSAGRTE 


' NO OriNION 

1 OR 

j UNDECIDED 


STATt ?/JENTS 


V 




1 




i 






\ ^ •_. f ^» ti . T. ^ A 

i:r r . uI.-i ' Hi fr^iii 
■ J. HUT I- . Tt..-fi\.. f. t'.vj. 



L Ir I ~tU, , .r..' t,- ■ 

) -^- 'I. if ,^ I..- ih.t 
I K. ^ Iv. ,U 'w tt,. .1 « ■ 
• 11. • , . J .u- . 

It.i I ; V .Kf.l jit.m .. L : ■ ir^ 



...M r.. .1 'aw. 

»ii.ir-f'r.' ;i u I i .,;.it;. ■ -I 

■••ir.Nt.n.t Ti-.t.-ii I .,„ 

. I .11. tit l.rrin.ili. t)f I 
1 ttl i:f.tv.;. 



Interest "Checks An extension program may 
succeed or fail in proportion to the emphasis which 
the extension wo* ker puts upon the developincni 
and encouragement of intere^its. Interests indicate 
likes and disKtve.^, s.i? icfucijuiicj md dissatistactions. 

Interest checks, Jevices for ilridui^ out in what 
the peoplj arc mterested, have been used to con- 
sitiorable extent in ^^Ixtension. 

The Suggested F(jrm fc^ an Interest Question- 
naire" illustrates a ■uothna of finding out what help 
ru:vd people ar^: intcrcjted in getting from the 
ccunLy agent. his is used as a partial basis for 
program plantnng. 
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Form for an Interest Qu^?5if::?,'^n.iire'' 

In the I'ollowing list of itijins-, the interviewer 
will check foi* each tlv:- rtogree to which the 
person^jnterviewed is i!")f crested in receiving 
informdtion or other assistance, and methods 
preferred. . 



Hem 



*. (Iruiirti/ut i.,t, f^^r cotinnxiiiity 
iinj'i "V » irii ' \ 



'. fl-'^k thfi.rni.itii. II nii cnns 
iiMi l:v>t(f-U 

... Mt'tU'Hls (.f jM-ttun- iriijiri'Vi- 
itii'iit 

, < J»<* ir« fru 'Ai r*' jfH hidv-I 
iti Hit*. li>i. > 



2. 



ORorce of Interest 
Check ( ) 



Much Some None 



Methods 
Preferred 



( Ml rM<-ii>>i<>i) »M<'rji- 
Will k.>--l ;is 
fnilows Ami rai li •'Un* 
>Mi-l.ltr)| fur )•< r><ill (<• 



-Mr. I i 



V- 

C <-lirnL,r 

8 — Ht^lhtin 
N^-^W-w;? niorv 



**How to Decide If you Are Interested in a 
Subject — 

"Much interest means: Ask yourself wheth- 
er you would read about it right away if you 
had the reading material, or if you would 
attend a meeting with other farmers to hear 
the subject discussed. If you would, then 
check in the "Much" column, meaning much 
interest. 

"Some interest means: If you are not sure, 
, or if you would pr^bably put off for some time 
reading about it dr attending a meeting to 
hear it discussed, then check in the "Some" 
column, xjaeaning sort^ejjiterest, but not much. 

'*If you feel sure you would not read about 
the subject or attend a meeting on it, no mat- 
ter how much t. ne or reading material you 
have, or how m:i :y meetings were held, then 
check in the "None" column, meaning no 
interest. y^' 

*'Be careful to consider and check ever^^^ 
statement, but do not spend too much time on 
any one statement." / 

have you lived in this communi^? 
years^ 



a. 



^0. 



3. How satisfied ate you with the way local gov 
ernment affair^ are handled? Would you say 
" yQu are not atpll sati.sfied, not very well satis- 
fied, pretty W^U satisfied* or very well satis- 
fied? 

( \ a. Very well satisfied 
• . ( ) b. Pretty well satisfied 



1 ) c. Not very well satisfied 
t ) d. Not at all satisfied ' 
( J e. i;nd.;:cided. 
Any cor Hionts: 



T! '? above question is\^e that has been used to 
api' .i.se satisfactions. 

Confidence Testing — Self-assurance or self-coi^fi- 
dence, the feeling that one can do a certain^fhing 
has in the past been consi<^ered too intangible to 
measure. And yet you ! _ probably observed a 
person and noted that "he is very confident" or 
"he lacks confidence." In so doing you have ob- 
served and d/awn a conclusion; you have "measur- 
ed" his confidence. Therefore, there must be tan- 
gible evidence of confidence. 

The following is a device or question which has 
been proven v^lid and has worked successfully. It 
can be adapted to measure confidence in many 
different kinds of things. 

Have you ever made a school dress for yourself?* 

' ( )*a,.Yes 
( ) b. No 

Do you think you can? (Check one) 

( ) c. No, I don't think I can. 

( ) d. Yes, I think I can, but am not sure. 

( ) e. Yes, I feel sure I can. 

Practices Adopted or Action Taken — Perhaps 
most of the devices and questions used in Exten- 
sion to measure change are aimed at finding out if 
the person has adopt^ed the practices taught. A 
practice adopted is also evidence of knowledge 
acquired and attitude changed. In time and energy 
management certain principles are taught such as 
"sitting to work" and ''eliminating some jobs." 

Following are examples. of two questions" taken 
from a check -sheet prepared by a home manage- 
ment specialist. This check sheet was designed to 
be used before and after the learning experience. 

'^^hfere are several different things which peo- 
ple do around the house. We are interested in 
finding out how many vvomen are doing these 
things. So I would like you to go through the 
list and check which ones you do." ' 
Home economics example: 

1. In ironjjogr^ you: 

a. Ij^n towels? 
Iron sheets? 

2. Do you sit to: 
a. Iron? 



UtuaHy Somettmes Never 

'.!) (2)_ (3) 

(1) (2) (3) 



(1) 



b. Prepare vegetables? ( '.) 

c. Wash dishes? (1) 



J2l 
(2). 
_(2) 



—(3). 
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Agricultural example: 

3. Do you sell"eggs on a graded basis? 
( ) a. Always 

{ ) b. Sometimes 
( ) c. Never 

4. Where are eggs kept until marketing".^ 
( ) a. In the house 

" { .vr b:" WhCTg— - — - - 

( ) c. Outbuilding (specify) 

Practices Adopted and Methods Responsible — 

Many of the .questions determine practices adopted 
and attempt to determine methods the respondent 
thinks are responsible. 

1. Do you have electricity? 

• . Yes (1) No (2).. 

2. If^es, a. Do you, ycurself, repair yoar own 
electric cords or plugs? 

Yes (1) No (2) 

b. If you repair cwn cords or plugs, where did 
you learn to do 'if ' . . 

Summary 

Devices for collecting data are forms on which 
the dat'^ can be recorded and used at a later time 
for anLiiysis and tabulating. Memory does not need 
to be depended upon. 

In selecting a -device we need to keep in mind 
the objectives to be evaluated and the evidence to 
look for. We have given some examples of ques- 
tions and forms for eight kinds of objectives. Many 
times, however, we must construct our own devices. 
This is the subject of the next chapter. 
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Chapter XIV 

CONSTRUCTION OF AN 
EVALUATION DEVICE 

Gladys Gallup 



The selection and construction of the measuring 
device depena.^ upon the kind of evidence wanted 
in light of the' objectives of the study. At this 
point the evaluator needs to think ahead how he 
plans to collect tiic information. 

Tho county ^gricui M agent may decide to use 
an observation'sheet t ry with him as he makes 
visits to a cross section of farms and homes. He 
may decide to send mail questionnaires to a sample 
of farmers or homemakers. The home demonstra- 
tion agent may develop report forms such as check- 
lists or questionnaires to use in collecting informa- 
tioa .regarding the adoption of practices from a 
sample of participating homeTnakers. 

Sometimes a field study is made. This is a more 
formul 'evalii^on study. It' requires a ^' fferent 
type of questior^aire or schedule as it is to be used 
to' interview a choss section sample of the people 
living in the county. You need, therefore, to con- 
sider not only the kind of evidence which you 
want to meet the objectives of the study, but also 
how you plan to collect the information. 

The questions asked should be so set up that 
information or data collected separately from a 
number of people can be added together or sum- 
marized to present a true picture of the group. 

The questions in the measuring device should be 
definite and specific questions related to the major 
questions which are to be answered by the evalua- 
tion. For example,^ if one of the major questions to 
be answered by the evaluation is '*What has been 
accomplished to control soil erosion on the farms," 
specific questions relating to soil erosion control , 
would be constructed for the questionnaire. 

When the replies to these specific questions are 
summarized, they answer the major questions on 
soil erosion control. The major questions are the ^ 
main purposes of the evaluation and the specific 
questions of the questionnaire are designed in rela- 
tion to the main purposes of the evaluation. 

Criteria for Constructing a Measuring Device 

For a device, such as a report form, an interest 
check, or a questionnaire to be effective there are 
five criteria which it must meet. These criteria are 



validity, reliability, objectivity, practicability, and 
simplicity. " * 

Validify is concerned with measuring what you 
want to measure. A valid device actually gets evi- 
dence of progress in the teaching objectives or of 
the iniormation you ^'desire. The two elements oi 
validity are behavior validity and subject-matter 
validity. 

Behavior Vc*Udity— A device has behavior validity 
if the behavior called for in the questions is the 
same as that expressed in the teaching objectives. 
The device gets evidence ' f the behavior you are 
teaching. Measuring skill, for example, will re- 
quire a different device from measuring knowledge, 
because of the different behavior involved, 

Subject-A^tter Validity — A valid device must 
deal with IJ^iS narJ.icular subject matter of the teach- 
ing objective U is obvious that if we are measuring 
attitudes totV>^."ci soil erosion control, we would not 
ask questions about attitudes toward using ferti- 
lizer. The device must contain the subject matter 
and situations in which the behavior is expressed 
as called for in the te^Tiing objectives. 

When a device is valid from the standpoints o5 
both behavior and subject matter, valid evidence 
can be obtained from which to draw conclusions 
and make recommendations about the teaching 
objectives, methods or other aspects of the program 
which we are trying to evaluate. 

Reliability is the degree to which the sample of 
•people or of subject matter represents and gives 
results which are like the results that would have 
been obtained from the whole population of people 
or of all the subject matter. It is a measure of 
whether the random sample is large enough to 
give sufficiently stable results. The sample may 
be a sample of people, a sample of subject matter, 
or a sample of behavior over a period of time, de* 
pending upon the "population" about^ which inter- 
pretations, are to be made. Exte^sion^research has 
been concerned mainly with samples of people. 

Objectivity means having a concrete, observable, 
and tangible basis for making a judgment or inter- 
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pretatioii. It is in contrast to subjectivity, ^ pinions, 
or guesses. An objective measuring dr /ice is so 
constructed that the results from it art iiol infUi- 
cnced by the difference in people who administer 
or interpret it. If equally competent people ask a 
question, interpret -the answers, and get closely 
similar results the measuring device is considered 
objective.. The use of numbers helps to/mako tter- 
pretations more objective. For example, ''100 per* 
sons'' is more objective than 'Mnauy persons.'* 

' •PfaHi'caBility— 

tical to use. It is necessary to consider time, cost^ 
and convenience. Will the foi'm take more time 
than it is worth; will it cost inore than you can 
afford?- It must be within your resources consider- 
ing all the demands made upon you. in pUuining 
the time schedule, the length of the questionnaire, 
the time involved in sui^imarizing it, the coopera- 
tion you will need of the'people and the probable^ 
value of ttTb results obtained continually must be 
balanced against its costs. 

Simplicity — Is the report form or -pjuestionnaire 
simple, easy to understand, easy to answer, easy 
to administer, easy to tabulate, and easy to sum- 
marize? The more simple a questionnaire, the more 
practical it w^l be to use and thg time between field 
work arid the report will be minimized. Simplicity 
s^'ould not be confused with brevity. 

The degree to whiofi a measuring device meets 
the criteria is conditioned by the 'circumstances of 
the study and the use to bCi made of the findings. 

Types of Questions 

In wording questions for the different measuring 
devices used in Extension simple types of questions 
have been used: 

Dichofomous Questions. Dichotomous means 
division into two parts or division by pairs. 

These questions present opposite choices design- 
ed to" get responsel^such as yes or no, agree or dis- 
a^ee, approve or disapprove. 

The chief advantage of dichotomous questions is 
simplicity. They take relatively little time to ask, 
and give a clear-cut answer which can be re\:orded 
and tabulated easily. 

The folloA^ng are typical dichotomous questions 
from extensioh>§iudies: 

Do you happen to know who is tl)e county agri- 
cultural agent? ' Yes No . 

^ Although these questions are designed as *'dicho- 
tomus" they usually should and do allow for a 
"third response, such as ''no opinion/' *'not sure," 
or *'don*t know." For example: 

During the past year have you ^attended: 



A women's extension club meeting conducted 
by the county home agent? 

Yes No Don't remember 

The chief advantage of the dichotomous question 
is that it can be answered with a concrete, definite 
answer. It gives a clear-cut reply which can be 
easily recorded and tabulated. 

One of the limitations of the "yes" or **no" ques- 
tion is that a slight misunderstanding of its mean- 
ing may result in a complete removal from a positive 

to £1 negative ans\ycr: -This^ may also increa^ 

**no answer" rate. This is especially true when 
getting a person's opinion. 

Multiple Choree Questions are used extensively 
in extension ^juestionnaires. A direct question is 
asked followed by a number of possible responses. 
The person is asked to select from several clearly 
.stated alternatives. 

Multiple choice questions are used when there 
can be several responses to a question. The person 
answering these questions is given an opportunity 
-^of choosing one or more of seyeral possible answers 
that are true, or i-epresent his opinions or ideas, or 
come closest to them. 

The directions may indicate that only the correct 
response should be checked or the directions may 
indicate that the respondent can check as many 
responses as he believes apply. 

Multiple choice questions are not easy to con- 
struct. The possible answers must be plausible and 
the right answer must not, be overly conspicuous. 
Example of multiple choice q[uGstions where one 
answer is correct: 

Which one of these plant foods put on as com- 
mercial fertilizer will *'leach'' or wash out of the 
soil most easily? (Check one only) 

V nitrogen _ 

phosphorous 

potash • _ 

calcium . 

An example of multiple choice checklist type in ' 
which one or more answers may. be checked: 

Which of the following methods do you use for 
keeping up with your farm business? (Check as 
many as you use) 



a. 

b. 
c. 

d. 



- Keep bills in a drawer, envelope, or some 

convenient place. ^ 
Keep check stubs. 

Keep a ledger (book) which { developed 

on my own, . 
Keep a farm cash account book (printed 

form). 

Keep in mind my expenses and receipts 

and add them up at the end of year. 
Other:^ _ ' 
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Open End Questions— rThese questions are usual- 
ly- used*in pers\)nal interview schedules. The re- 
spondent is free to answer as he likes. Open end 
questions give such a variety of responses that the 
job of classifying and analyzing is both time con- 
suming and difficult. 

Open end questions are often used in pretesting 
to get a line orr^* possible answers for an. eventual 
check list type of question. 

When it is necessary to employ questions which 
give the respondent free latitude in his responses, 
-?P9!l.^^E!?..5Ly.^^M?.'??^ _jEx_arnple: 

1. How does your milk production in October, 
November^ and December compare with other 
months of the year — is it higher, lower, or 
about the same? (Circle one) 

IF **HIGHER" OR "LOWER** 

2. About how much (higher) (lower)?. , 



(percent) 

IF FALL PRODUCTION IS SAME OR HIGHER 
3. Many farmers are not able to keep their pro-' 
duction up during these nionths. How do you 
manage to do it? 1_ 



4. Where did you get the idea? 



IF FALL PRODUCTION IS LOW COMPARED 
WITH REST OF YEAR 

5. Have you ever heard of the idea of making 
your milk p^0^^uction even throughout the 
year by having '^ows freshen at different 
times? ' * Yes No 

6. If yes, what do you think of the idea?^, 



Wording of Questions 



The wording of questions affects" the answers. 
Each question must be worded to give exactly the 
information desired. Go over each question and 
ask yourself these questions. 

1* Is the question related directly to the purpose 
of the study? 

2. Can there be any doubt about the definition 
of any word in the question? Will each re- 
spondent understand each word as you do? 

3. Are the words in each question familiar to 
the person who is to answer? 

• 4. Are ^questions simple, short, clear, and con- 
cise? 

5. Have you avoided asking leading questions — 
questions worded in such a way as to suggest 
answers? 



6. Have you avoided ''double-barreled** ques- 
tions, questions that have more than one idea? 

7. Have you avoided doubite negatives? 

8. Are definite time limits on questions specified, 
for example, ^'During t past 12 months?'' 

9. Has a check yes and no answer been used 
whenever advisable? 

10. Is there an opportunity for the respondent or 
cooperator to indicate that the question does 
not apply to him or her? 

Face^F D^scnpTtive D¥ta ^ 

Besides the questions relating to the objectives 
of the work being studied, additional information 
niay be needed about the respondent which will 
help in interpreting the answers-— such itehis as 
cumber in family or place of residence. 

Certain "face data*' questions which can be 
checked against census data have been used in 
extension studies. These include: Formal school- 
ing, place of residence, age groups, total net annual 
family income, and certain items on the Sewell 
level of Uvii:g scale such as running water in the 
home, complete bathroom, and other conveniences. 

These face data questions provide information 
by which the study data can be classified. For 
example, we may want to know if the people who 
adopted certain practices live on a farm or not, 
if they are older or youngen. or 40 years of age and 
over, or under 40 years of age, if their formal 
schooling was 8 years or less, or more than 8 years. 
(A much more detailed classification ma^ ' e 
desired.) By classifying the information in ir? r 
way we can find the relationship between different 
factors and changed practices. Examples of face 
data questions: 

- Please check the HIGHEST grade you completed 

in school: 

Elementary school 

a. ' 1st to 4th grade 

b. 5th to 6th grade 

' c. 7th grade 

d. 8th grade 



High school 

e. 1 to 3 years 

f . 4 years _ 

College 

g. 1 to 3 years 

h. 4 years . 

i. more - 

The questions of this type that are to be include^ 
in the questionnaire depend upon the pur^^ose oY 
the. study and how the data are .to be analyzed. \^ 
Other examples of face data are: 
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Where is your home located? Find thr one of 
the seven following answers which best describes 
whore you live. (Check one only) 
• a. On a farm or ranch 

b. In the country but not on a farm or ranch 

c. On a farm or ranch part of the year and 

in town part of the year 

d. In a town or village with less than 
2,500 people 

e. In a t(}wrT or city \s'ith 2.500 to 

10,000 people . 

1*. In a town oiLcjLy u'it^^^ than 

10.000 people 
g. In a suburban area (not in city limits) 
of a city with 10,000 or more people 

In which age group do you belong'' Please find 
the age group below in which you belong. (Check 
one only) 



19 ye; 's or under 
20 — 2 i yeai s 
25—29 years 
30—39 years 
40 — 49 years 
50 — 59 years 
(50 vears or old^M' 



Please indicate which of the following is- nearest 
your net income after farm or business expenses 
are deducted. <Check one onlv) 



a. Loss than $1,000 
b $1,000 — $1,499 
.V 1.500 — $2,499 
. $-^500 — $3,499 

■Core Questions 



e. $3,500 — $4,999 
£.55,000 — $6,999 
g. $V.OOO — $9,999. 
Ji. $10,000 or over. 



There is a series of questions that has been devel- 
oped and u:;ed in extension studies to determine 
the krovvlecJge which people have of extension, 
k'vol of living items, and face data C|uestions. 

There are many advantages in using these core 
questions as they have been pretested under differ- 
ent conditions and are well designed. Most impor- 
tant, they provide coniparable data because the 
(juesMons are identical in wording. This insures 
thai the results of a study will be more meaningful 
because Ihey can be compared with the findings of 
other related studies. 

Ir. addition to tho face data (juestions, a series of 
questions has been developed to determine knowl- 
edge of and participation in extension. Examples 
of such questions which deal with 4 H Club work 
are: 

you know something of 4-ir Chib work? 

Yc>s . No 

Ho you have children in 4-H Club work? 

Yes No 



Have you ever had children in 4-H C1u}d wo -k? 

Yes_k..rc. 

Have you yourself ever been a member or leader 
of 4-PI Club? • Yes_. -.No 

Another example of a core question as used in 
home economics nutrition studies is: 

Do you yourself drink milk? About how much 
each day? 

a. One glass 

b. Two or more glasses 

c. None 



Sequence of Questions 

Logical Order — Place the questions in the logical 
order so that i\ person reading them does not have 
to jQmp from subject to subject in^ order to answer 
them. When the data are to be obtained by the 
intervie^iiethod, the questions should be grouped 
so that the conversation will lead logically from 
one question to the next. 

If general and :>pecific cfuestions are used, the 
general questions should precede the specific ones; 
otherwise, the respondent is likely to answer the 
general questions in terms of the earlier specific 
ones. For example, a genel-al question as '^During 
the past year what, change? have you made in your 
house, such as remodeling?'' should precede the 
more specific question, **During the past year what 
changes have you made 'in the kitchen?'* You do 
not want the respondent to think about kitchens 
only and not mention other more general improve- 
ments: 

Psychological Order-j-You can ha' e a logical 
order of questions but tne opening question jsjiould 
also l'u.\c human interest appeal. Th^v.ope'hing 
questions should be easy to answer and of interest 
to the respondejpt. 

A few simple questions at the beginning of the 
questionnaire will give the respondent confidence 
in himself. This is invaluable for carrying the 
interview through later and more complicated 
questioning. Questions of personal nature, like 
asking lor the age or income level of the informant 
should be placed near the end of the questionnaire. 

In designing mail questionnaires the arrange- 
ment of questions in order of interest is particularly 
important. 

Mechanical Setup 

Four Parts to ^a Questionnaire — Each well-plan- 
ned device or questionnaire has four parts: 

1. Identifying information such as name of ques- 
tionnaire or study, county, date, and name of 
person filling out form. If the study is of the 
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more formal type in which schedules are used 
by personal interviewers, the schedule should 
also include number of survey area, name of 
interviewer, and place for checker's initials. 
Also the schedule should include final dispo- 
sition such as: O.K., refused, incomplete but 
closed, etc. 

2. The introduction is important because it tends 
to orient the respondent to the p^irpose of the 
survey and tries to nwtivate answers to the 
questions. When records are gotten by mail, 
motivation is especially important. The open- 
ing sentences should explain: 

a. By whom the study is being made. 

b. What is the purpose of the study? 

c. To what use the information will be put. 

Adequate response depends largely on how well- 
stated are the facts of the introduction. Any re- 
spondent iS' due the cpurtasy of an explanation 
^covering the above points. 

3. The content questions based on objectives. 

4. Face data, if face data are necessary. 

Physical Arrangement — The following six sug- 
/gestions are on physical arrangement: 

1. Number each item consecutively. . 

2. Never repeat a number on the schedule. This 
facilitates reference to items where instruc- 
tions and definitions are given. 

3. Have each item clearly distinguished for tabu- 
lation purposes. 

, 4. Vary the type within a question to emphasize 
the most important words or phrases. 

5. Make form easy to read and fill out. Have 
places fpr checks rather than for words even 

-V ns short as yes and no. 

6. Place spaces for such checks in vertical col- 
umns ahnig the edge of the sheet to make 
summarization easier. 

Adequate Space — When you expect information 
to be written in, provide adequate space for- it. Do 
not expect people to use margins. 

Good Appearance — The form should have good 
appearance. This can be accomplished by good 
paper, large print, wide spaces,. and good arrange- 
ment. 

Special Rules for Constructing the 
Mail Questionnaire 

1, Appearance. , The questionnaire should be 
attractiyo. This can be accomplished by ap- 
propriate use of colors, pictures, spacing, type, 
arrangement, and so forth. 
. 2. Length. The questionnaire should appear easy 
to fill put and be as .short as possible. 



3. The letter of transmittal or cover letter sent 
with the questionnaire* should have a personal 
touch and an appeal which seems likely to 
bring the highest proportion of returns as 
indicated in the pretest. 

4. The letter should be frankable. With each 
cover letter and questionnaire there should be 
an enclosed f;anked envelope addressed to the 
extension worker for the respondent to return 
the questionnair e. . 

Pretesting the Questionnaire 

Every device used in measuring should be pre- 
tested. The way to. l^arh whether the test is ready 
for use is to try it out. The pretest' helps^.you to 
know if the content of the questions and the form 
of the questions are satisfactory. Then; aK(5 two 
important points: 

1. Try the questionnaire with persons who re- 
semble those with whom you are finally going 
to use it. 

2. Test the questionnaire under conditions simi- 
lar to those in the study. For example, when 
pretesting a mail questionnaire have persons 
answer if without help from you. 'Tell them 
you have a '^'uestionnaire you wish them *to 
try out ani! i they have finished ask for 
opinions ana f^estions. 

From a careful pretesting you get information 
about the following: How long it takes to fill out 
the questionnaire; whether the questions follow 
each other in natural order;- whether ample space 
i3 provided for answers; and whether the directions, 
are understood. 

You have to determine how widely yo^i pretest 
the questionnaire, and how often you revise it", in 
terms of importance ^f the study, the precision you 
need, and the amount of time and effort you can 
afford to spend upon the questionnaire. 

Summary ^ 

The questions in the evaluation device should be 
definite, specific and related to the major questions 
to be answered by the evaluation. For a device 
such as a .questionnaire to be effective~^there are 
five criteria which it must meet. These criteria 
are validity, reliability, objectivity, pi'acticability, ■ 
and simplicity. 

The wording of questions affects the answers. 
Each question must he' worded to give exactly tlic . 
information desired. Special rules for constructing 
questionnaires should be fallowed. ^ 
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Chapter XV 
TABULATION 

Ward F, Porter 
il//*.s\ Jewell G. Fcsscnde^ 



Tabulation is the process whereby study data are 
organized or grouped according to plan so as to 
facilitate "analysis and interpretation. Most of us 
— ti5e~the-birn pier-tabula tiiTg-oi^tallying-t^ehniques- 
quite often in our day-to-day extension tasks. We 
note the number of incoming and outgoing tele- 
phone calls that are made at the office. We keeo 
a running total on how many publications we distri- 
bute and what kinds. We have a tally sheet for 
office calls, farm visits, meetings with'^ extension 
groups, liaison actions with other organizations^ and 
the like. We have a frequency distribution of the, 
topics we deal vv.ith on our radio or television pro- 
grams or in our newspaper or magazine columns. 

When it comes to evaluation work and. research 
in general, it is obvious we could not get the job 
done without resorting to tabulations We might 
not have, to work out elaborate tally sheets or in- 
volved coding plans, but we must do so'me counting 
or we can make no pretense of having done an 
objective study. ' \ 

Planning for Tabulation f^' / 

We should work out our^STrcTlation plans prior to 
th^ completion of the questionnaires or other recoi d 
forms. We have already pointed out that we need 
to know what information we want before we can 
buiFd a schedule or questionnaire to secure it. 
Thinking about the- tabulation should properly com- * 
mence after we decide what information vve want 
and while wc are working out practical and objec- 
tive ways*to collect the necessary data. 

In planning for tabulation, we should keep four 
major considerations in mind. ^^^^ 

1. Wha9- Resources and Skills Will Be Available 
to Us?^ We cannot plan for a very comprehensive 
tabular analysis if we nm.st have a quick answer or 
if there will be limited cic rical and secretarial help. 
If we are involved in the kind of study wherein we 
have to make our own analyses at odd moments 
stolen from our regular tasks, we have to be satis- 
fied with relatively brief and simple study designs, 
record. forms, and easily tabulated answers. 

'Where machine facilities and operators are avail- 
able, we can plan much more involved and elaborate 
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tabulations and can work ,out extensive cross- 
tabulations. In such cases, it is essential to include 
the machine technicians in our planning. Since 
— many-sGhGd^aes-Xii:e-iApi!e--coded,--Ltlie-assistanco--of 
machine operators may be required well in advance 
of actual data collection. 

If we will have to do hand sorting and counting 
'with or without assistance, we cannot hope to take 
on so large a program of analysis as we would if we 
could count on electronic equipment. The problem 
that could be wjorked up in a few days with a trained 
crew of machine^ operhtbrs might take months,. or 
be impossible to accomplish, using, hand methods, 
before the findings would be obsolete. 

In addition to physical facilities, we have to con- 
sider certain hun^an factors, such as ouV own abili- 
ties and those of persons who will be at our disposaj 
for the study. If we have no competence in statis- 
tics and have difficulty in following through on 
involved relationships, we may have to settle for 
reporting totals, working oat a few percentages, 
and possibly calculating some averages. If we have' 
some competence in such matters, or someone is 
available to guide us, we can work out a more com- 
pre'hensive plan of analysis-ranking, correlating, 
intercorrelating, testing for significance, validity, 
reliability, and so on. 

2. What Information Will We Need for Our 
Analyses? The purpose of the schedule or ques- 
tionnaire, ' of course, is to secure the information 
we need. However, we can be more certain it will 
secure the information we need by drawi^ig up some 
tentative tally sl\eets and work tables, preferably 
before the final draift 'of the' schedule. This -will 
help us see, for example, whether or not the prob- 
able answers will be in a form or of a type that will 
facilitate the analyses we wish to. coijduct!' Once 
wo are satis;^ied in this regard, we can make up the 
final tabulation forms, and be ready to make entries 
as soon as the data are received. 

We must* remember that we cannot plan for an 
analysis that required more detailed information 
, than can be supplied by our record forms. For 
instance, we cannot make year-by-year breakdowns' 
in our tables if our age data are collected in terms 
of five-year intervals. Nor 'can we readdy report 



. ' [70] 

ERIC ' S0 



average or .mean income if we collect income data 
in terms of categories such as under 35.000. 35.000 
through $9,999, and $10,000 or over. 

On the other hand, however, we can make our 
tabulated information less detailed than that i eport- 
cd in our source. Wo may collect age inr rniation 
by age at last birthday and report 'it li ur tally, 
sheets by five-year intervals. We may have a com- 
plete age-sex breakiclown but report in terms of 
numbers of adults or children per household. We 
may secure actual income data (very difficult to do) 
but report in :i . frc'quency distribution by S5.000 
intervals. , 

3. What Relationships Do We Wish to Study? We 

must decide on .\v hat relationships we plan to study 
"~STJHrat~wc^lTaw\the~proper questioTi5"in"our~data- ^ 
gathering instrunnent and have adequate provisions 
in our tabulating, forms fo record the relationships 
in categories suitnble to our needs. 

To illustrate: lA we want to know the differences 
in interests of respondents by age, educational level, 
place of residence, and sex, we must be able to sort 
on thcso items and then tabulate in terms of their 
answer.s regarding interests*. In our tabulation 
plans we must decide what age, residence, and 
education ^categories will serve our purposes, ancj 
then be stire our source reports the information in 
enough detail to set up these categories. j 

4. What Comparisons Do We Expect to Make? 

Whenever we Intend^to compare certaili data from 
our study with thart of other studies, we must be 
certain to collect the necessary information and 
assemble it in such a way that there is compar- 
ability. Quite frequently we like to check on the 
probable representa,tiveness of our\sample by mak- 
ing coniparison^^vith census or other statistical 
information r^Jated to our study area. Wl'ienever 
this is our intention, w^e must be sure to have our 
backgjound information (our ''face data") defined 
and classified so as to match that used by the cen- 
sus, or other sources to be used in'our co'mparisons. 

This is a strong argument for collecting our basic 
information in }uite detailed fashion. Then we are 
at liberty to make a wider variety of tabulation 
.decisions. We can group our data in the conven- 
tional categories used by the census and similar 
governmental agencies to make comparisons with » 
their findings. Then, if needed, we can make 
special groupings to permit comparisons with find- 
ings of special studies by colleagues in Extension 
or elsewhere who have done work similar to our 
own, • ' ' . , " 

Another common occasion for comparisons is in 
before-after or trend studies.* When you are 
studying the change that has occurred since some 
previous reporting date, oi* when you are observing 
the effects of an action program, you have urgent 



need for comparable data. Whenever this is your 
objective, you must ask questions and prepare tab* 
ulations in essentially the same, form for both 
occasions. 

Vou cannot ask for annual family income one 
time and annual farm income the next. Nor can 
you ask for income to the nearest $1,000 one time 
and then decide to simplify the item next time by 
setting up S5,000 income categories. 

Tabulation Techniques 

Tabulation consists of listing or grouping answ'ers 
from all respondents, or from all members of par- 
» ticular groups withm the population being studied, 
._-So.tliaL^.uminary_ansAVcxs„can_be-obtained.I Possible 
techniques for assembling these sunimary, average, 
or consensus answers vary widely. The ones you 
select will depend on the kind of study you are 
doing, the resources that arc available, and what 
you want to get^ut of your data. We will mention 
some of the techniques that- you might use in 
extension evaluation. 

Tally Sheet — This is simply, a counting aid, al- 
though it can be elaborated into a sorting device. 
On a card or sheet of paper of sufficient size, you 
^set up column headings according to the categories 
into w^hich yotrwish to sort or subdivide your data. 
Down the left side of the page you list the possible 
answers to the question. 

^* County ^ 

Person(sHalIying 



bate 



HAND TALjiY SHEET (Example) 

- : Community Development Survey 





All 


Total 


Total 




household 


farm 


nonfarm 


7. Level of liviii^i: 








Total hoii.seliolcis 


44 


25 


19 ^ 


No 'onsc 


3 


1 


2 


Reporting 


41 


24 


r 

17 


• a. Electricity 


^ 40 
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b. Telephone 
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iHifHina 


c. Running water 


25 
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d. Central heating 


19 


( 6) 


(13) 
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twmiui 


e. Electric washer 


27 


(i3> 


(14) 






/Ml ft4iire 




i f. Nono " 


0 


( 0) 





In.AOur example, the sorting factor is residence 
(farm or nonfarni), while the question concerns the 
level of living of the families in the sample/ 

We will' assume the respondents (or the intei- 
vieu'ers) were asked to check cuily the facilities 
which were available in the homes. 

Since i't is necessary to account for each and every 
respondent, allowances have been made for non- 
responses, and for households reporting none of 
the items listed. This *'rule" applies in gdheral to 
all tabulations.' For certain types of questions, there 
might be justification for still another category — 
that is, one to account for respondents, who report 
not knowing or *'no opinion" or ''don't know/j Tho 
importance of this consideration has- already been 
stressed in our discussion of questionnaiie con- 
stiuction. 

To construct such a tally sheet, you reserve space "* 
the top for identifying information. Depending 
on the circumstances, such identifying data might 
include: (1) working title of the study. (2) working; 
title of the partkular table (tally sheet), (3) the 
name or initials of each person working on the 
sheet, including counter, reader, checker, and any 
others, and (4) date work was completed. It-.is a 
good idea to type the column'^ and row headings 
and draw in necessary guide lines. 

You can make- the tally with a typewriter, if you 
have a wide-carriage machine,. by having one per- 
\ son Vead off the numbers .and where they^should 
}go, aiid the other enter them with the typewriter. 
^ The niore conventional way is to draw up the tally 
form and then make the entries by hand with pen 
or pencil. You can still m^ike typewritten copy for 
your permanent record, however. 

In tallying, it is customary to make four perpen- 
dicular marks and then make the fifth one a slash 
through the previous four * This facilitates counting 
by fives and makes it easier to observe relationships 
by inspection rather than by actually counting. 
'Subtotals can be placed inside brackets at the ,end 
of the count or immediately above or below. 

Thi3 technique is most effective when only. a few 
records are to b^. Rallied, and when there is not a' 
great amount of subsorting. The greatest disadvan- 
tages of this technique arc the likelihood of error* 
and the difficulty in checking for error. About the 
only way to check'the work is to-do it over, or have 
someone else do it over and compare results. Even^' 
the same results do^not rule out the possibility that 
counteracting errors have been m'ade. The pros- 
pects for error increase hiaterially. if there are 
" interruptions during the tallying process. 

Although tally sheets are most useful in limited, 
.small-scale studies, there are at least* two occasions 
\vhen tally sheets have a place in*^thc more complex 
Evaluations or surveys. * First, they can be used to 
advantage' in taking off key items of interest in a 



preliminary, tentative, unchecked sgrt of way. Such 
data should never be reported or published, but 

. they can be used to get an idea of what the results 
will be. : . . ' 

A second occasion where they can be useful is 
when the researchers have only vague ideas of how 
certain data should be classified or categorized. 
This occurs often with" essay type items. The 
clerks can take a portion of the records, note each 
reply as it occurs, and mark down a tally for each 
time it is repeated. Soon a pattern emerges and 
you can set up sufficient broad categories'(column 
headings) to take care of most of the. answers you 
' are likely to get. Any others can be scored jn the 
'*other'' column. At this point, the researciher is 

<^ in a position to work up a code for use in some one 
of the other tabulating techniques^ we will mention' 
shortly. • ^ ^' 

. ' . •(- . : 

Handsorting — This is another counting device 
which works quite >yell with small amounts of data. 
Instead:pjf rioting the answer and'entering it on your 
lally.^eet, you sort the answers into agreed-upon 
cat^egories' by stacking up record forms into piles 
containing the same or sufficiently similar answers. 
Then you count the forms in each pile to get totals 
for your record sheets, frequency tables-, or charts. 
An alternate approach, where the. record forms are 
lengthy or^'bulky, is to transfer «the. questions and 
answers to, special answer cards. Both questions 
and- answers can be coded or abbreviated to save 
space. - ^ 

The majoV advjintage of tht*' method is its^sim- 
plicity. You need no special equ^^ment to do the 
work; and no special skill is rciyii'ed to do the 
sorting, although there is ahvays a need for dis- 
crimination when sorting essay type answers. . 

A major disadvantage is that you* need quite a 
large working space and relative assurance that 
the piles will not be disturbed. Another Js that the 

task becomes quite involved if the record form is 
]i)ulky. Turning to the proper page for the sort 

. item, and then to the page for the question you are 
analyzing, can become irritating arid is very time 
consuming. However, if your record form is a 
one-page, affair, and especially so if it is on a card, 
you should have no' difficulty with this phase of 

' Ixandsorting. ^ ^ - 

In other respects, the advantages and disadvan- 
'* tages of handsorting are quite sinjilar to^those asso-c 
ciated with the tally sheet approach to tabulation. 
Both* methods are more subject to error than' the 
other procedures to* be discussed. On the other _ 
.hand, both handsorting and tallying are very prac- 
tical -and elective techniques for determining 
appropriate cllissifications and for preliminary 
reporting of selected data from., surveys 'and other 
formal studies. » ^ 



TabuUtion Sheet — An (.'labfinuioii of the tally 
sht-et. : "mI th'' aiisurr card as \\v\\, is the tabulation 
sheet. This iMiisists oi a ennsolidateH listii^u of 
answers to ali Us;ibl(» ((uestif)!is on all records in the 
study Vhr uwihod is most time eonsinnin^S4 but 
tiiere are rnaiiy opportunities loi* eheckin^ on the 
u or"k, and \ ou can use the I inal pro(hiet Tor buildinj^ 
u'liatevo!' tables you uesii'e. I rdike tlie tnliy sheet. 
>un ean find errors nioi'e easily since i*ac ii (p.u»^- 
tifjnnaire is identified f^:. slieet. 

'I1ie sL'tup is nuieii like a tally sheet, \\ i\b siii able 
eohunn hea(lin.t;s auvl subh(\ulin^s Likewise, ytni 
ear] make y<jur eiiti ies with a I vpewi i(ei' or by hand. 
Ih)we\-er. the foi'mat diffei' in th;it what >i,u Wf)uh! 
nut in tlie niarL^in or "stub" o!i a tally sheet is usual- 
ly im lufU'd Uiiderthe inajoi- lolunm ,tiea(!\:if;:; across 
tho top Karh hue or '"row ' across the .->h.'et i erjorts 
mf'^rrnation f!'(Mn a sinyk^ vf-'nnl form. Fer an 
illusti'ation cjf the format i>i a sinii^lj tabu! ion 
sheoi. see tlie S'deetion' entith'd, "KvaUialing a 
['astute, fmprovement Ihcj^ram." aopendix A, j). 
100. 

A ;4rr'aL (h.-al of thf)u.i4hl nuist be given lo our 
,:MJilen(h'(| use of the (hda hefoie we set up a tabida- 
tion slieet. Althr)U<;h we do not have to enter all 
I he infoiTiialion w e iutve collet ted, we sliouhl be 
( arcdid to plan f(jr inehision of all we aj e Jikely lo 
nei'd for our analysis, Once vve ha\e planned tt^e 
forntat and or^ani7.atif)n of the tabulation sheet, 
and (■(>mf)!eted our tabuhition, we cannot reline our 
data lo inchidc more detail However, if we allow 
for all eventualities, we ean consolidate calegorii ,:, 
wiiene\er that seems advisable. Ftjr this i-eason, 
ue shr)uld make our original entr ies as detailed a.^ 
ue are likely lo have need for* them to be. What 
may seem like uimeeessary extra\ a.^anee of |)a[) r 
and effort at the tinie may be an econt^ny befoi'e 
'the analyses are eon^pleted. 

In plannin^^ our tabulation sheet. v;e njust also 
consider all of the j^ossible resprnises to each (jues- 
tion and plan our colunms and headings ca-eordin^- 
ly. Vol instance, where l espondent cindd answei* 
'•yes," "no/* or •"don't kumv" to a f|uestion» or 
refuse to ansxver it entirely, we might propei'ly 
allow a cohimn for each of these four possible type's 
of respon.ses. This would give us a complete 
aceoimtin^ of all the i-espondeiUs. 

hi our example of a (abulation sheet ^appendix 
A. p 100}, notice that cpiestion .zrf) has three pos- 
sjhU.' answers: in} '"within past 3 years," (b) 'Mnoi-e 
than :i years a.iio/' and u. "'dotrt know'* i "l),K."i. 
In addition, we alloweci one ad(Utional ''{duum for* 
noni(\sponse to the (pK?stif)n, 

In sf>me instances, one or" more (j u dior»s in a 
schedule may not api)ly to some of the respiHulenl,-.. 
In such cases, a cf)lumn eaj)tione(i '^/'A' (*"\ot 
Ap|)ly*') might he added, Kefeirinj; a^ain to our. 
exam|>le in the appen'lix. we could qiwte propei ly 
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add an '".XA" c;(/l :;mn imder (jue>tion z=2. This 
would then ru-eessitate an enli-y in the "\A* ceh 
umn f(jr aK lespondents who had not limed their 
l).-j.-rjre <e..ti.. (|uestionnane —2 and —4 in our illus- 
lr;.'^'orji 

To .-.epa^ate the data into groups for comparison. 
tlH' records must be classified before tabulation 
sr:-ts. This indicates !he necessity oi knowing. 
'^<*rore you start t:if)n]ating, uhich comparisons yoM 
wish to make fiat kinds of .results you will want. 
The records ean be broken down into any nimiber 
of subgroups, and the various subgi'oup totals add- 
vil together in various condjinations to get various 
group totals. This is, admittjdly. a cundoersome 
method of getting group compa!"i^()ns and relation- 
ships, fjut it i.^ effet tive and po. if>ie when electric 
maeliine tabulation is impossible. 

Machine Tabulation--Un!ess you are iniei-ested 
in <iuick ansv.ers to a few sim|)ie fpiestions, and 
plan U) make no intensive study of the answers. 
jiKiehine tabulation generally is the preferred 
metluxl of tabulation. 

However, this method i-equ.rcs access to some 
\ery expensive machinery that will not be available 
to the extension worker for routine evnhiations; so 
we will mentioi^ only .some of Mie characteristics of 
the method in this nianuah In any event, if any of 
you become involved in in evaluation where this 
method should be used, and facilities can be made 
available to your research team, you will want to 
consult with the operators and supervisors of the 
installation where the data will be processed. 

Most land-grant institutions now have such faeiii- 
lies. ICxtension researchers working on approved 
projects generally ean make arrangements to have 
their data machine-processed. Persons responsible 
lor the use ot the machines should be contacted 
eai'iy in the stiuly. 

These people can advise you on setting up your 
i t^search instrunients l.u seciu'e data most readily 
pi"ocessed by n^achine opei-ations, and they can 
alert you to the kinds of anai'y.T's you can make. 
They can assist you with coding" problems and can 
show^you ways of dealing with free-answer infor- 
niiit^». . They ean .suggest ways of using machine 
()I)erations to speed up statistical analyses by pro- 
viding totals ready for processing in calculating 
itiaehines. For that matter, some of the more elab- 
orate machines can perform basic statistical opera- 
tiort.s lor you. 

Tlu- immediate co.sts of machine processing are 
s{)metinies ()uite impressive, however well-justified. 
For this reason, there should be a clear understand- 
ing of costs befoi'c either thc^ researcher or the 
machine suj)ei'visor conmdts himself beyond recall. 
They should agioe on what operations will be re- 
(piired, and how many. They should agree on the 
coding, manner of transcribing to machine record 
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forms, amount of chccki and sequence 

of sorts and tabulation i of c Iculations 

•iijcl correlations to be do ,0 on. 

There are various compaiues making different 
types of machines^ for tabulating and summarizing 
answers frum record forms or questionnaires. Sojue 
questionnaires, set up for certain machines, are 
marked with a s'pecial kind of pencil so that the 
forms themselves can be run ihrough the machine. 
The pencil marks make electric contact for a count- 
ing of the answers. For other machines, every 
answer on the record form is reduced to a numeri- 
cal or alphabetical codq; then these codes are punch- 
ed onto small cards. Coding must be planned by 
someone who thoroughly understands the study. 
The cards are run through sorting machines which 
sorts out the answers to any questions we want,' 
;:;iving the number reporting each type of answer.- 
Or the tabulating machine totals the entries, giving 
us the sum for all the questionnaires, and the num- 
ber reporting Uie answers. 

If the questijOnnaires. are set up so that the 
answers , cr. .rked are assigned predfetermined 
numerical va-ues (i.e., precoded), the work and time 
involved in electric machine tabulation is less than 
for typewriter tabulation, and there is less chance 
for human error. The great advantage, however, 
of electric machine tabulation is the increased free- 
dom in manipulation of the cards. Groups do not 
have to be all predetermined. If an interesting 
casual relationship \r: indicated in early sunimaries^ 
of the data, the cards can be sorted oh the machines 
immediately, with no necessity for a complete 
retabulation as is n^'cessary with typewriter tabu- 
lation. 

Manual Tabulation Using Punched Cards — Sys- 
tems have been developed whereby data are coded 
and punched on 'ards that are sorted manually. 
Such a system is useful when the numbei; of cases 
ipvolved is limited. Extension workers can usually 
secure help and advite on the use of such cards 
from college statistical units. 

Free-Answer Tabulation — Classification and sum- 
marization of answers to open-end (|Uo:«|ions are 
difficult. Mapv times it is impossiblq to fiiake a 
usable classificCion system until the vSnswers arc 
at hand so they can be studied as a unit. 

If you have a relatively small sample, you can 
have all the answers to one question typed, out on 
a single sheet and determine appropriate categories 
by inspection. What generally proves easier, how 
ever, is to enter each answer to the question on a 
card just large enough to hold the informati<3hi. 
Thepi you start grouping ^elated answers and cpn- 
solidating related grouj^^s until you hav^-^wfSable 
number of categories. Now you can assign a code 
to the categories and use it to make entries on 



answer cards or other u^h Mating devices so that 
you can use these free-an,>wcr items as readily as 
any others in sorting and tabulating. 

If you have a large sample, you can save time but 
still get the job done by goi'^ij thj'ough the same 
process, with a subsanij^le 0: your records. The 
first fifty questionnaires :night give you the idea. 
On the other hand, you might prefer to take every 
• nth" record. 

Quite often, in the course of developing cate- 
gories, you will end up with an ' other"* or *'miscel' 
ianeous'* category. This will accommodate those 
responses that do not quite fit into your main 
classifications or groupings. This wili not detract 
from your analyses unless too many of your rec^ords 
wind up in this ;*other" group. Any time it becomes 
your largest category, youshouk^ onsidf"'- redesigYi- 
ing your classification system. 

Because so much of the i-ltin; :;e effectivenes.s 
of your study depends on how well you haiidle youv 
open-end, free-answer, or jssay-type questions, it is 
essential that you make decisions regarding them 
with [he greatest care. This is an occasion to con- 
sult with subject-matter and reseanrffYpecialists to 
be sure that you are making prcp^^r decisions. You 
may everr^vant to set up a pane^x)f expertsT^o>^work 
out a classification ^tem for certain items. 

This difficalty is the major reason why you should 
generally restrict the use of iree-answer items. 
Adequate pretesting should sug^st ways to set up 
check-off questions to take their pJace. Of course, 
there will be times when elaborate pretesting is not 
posf=!ble or when you ar^^ exploring a new^ topic 
where you have no idea of what the answers will 
be. In such cases, f rde-answer questions are a must. 

Another problem lies in the fact that check-off 
questions limit the range of possible answers. This 
may contribute to answers which are the result of 
snai judgment or ''any answer to get on with it" 
reactions. Even if . you provide an **other'' cate- 
gory, respondents cannot always be counted on to 
volunteer information. On the other hand, you 
cannot be sure how complete your replies to free- 
answer items really are, largely because of this 
reluctance to volunteer information. 

In one case you may have codeable and statis- 
tically manipulatable data of uncertain validity; n 
the other, ycu have had-to-code, hard-to-analyze, but 
possibly more valid da1;i. You will liave to decide, 
in ter- is of The particular item, which type of q»i?s- 
lion' w?'\ Ovovidc iic kind of answer that will be 
most elic .live fo: your purposes. 



Summary 



Our concern in chapter 'las been wHth ways 
of converting ma.',i,t'>j of Mifoiination into* under- 
standable and usable fbrni to^aid in analysis, report- 
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ing, and decision making. We have suggested that 
labulaticm must occupy an impo]*tant spot in re- 
search and r^- aluation. and that it is most effective 
when it is* planned at the same'^time the other re- 
s'carch instruments arc planned. 

There are at least four points to consider in plan- 
ning tabulations; H) available resources, (2Xinfor- 
mation needed, i3\ relationships of interesl to the 
planners of the study, and (4) comparisons 'desired. 
We have suggc\^*ed that, v/hen in doubt, it 's best 
to collect data in a more detailed ^orm than you 
are likelv to use. 



We have described some of the common tabula- 
tion techniques, ineiuding tallying,^ hanclsorting. 
tabulation sheet, machine processing, and manual 
. tabulation ' using punched cards. The essential 
chai acteristics and the relative advantages and dis- 
advantagiJs of each method ar.e br !fly discussed. 
The d^aptei' concludes vith some suggestions on 
the^J^andiing of free-answer responses. 
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^ \ Chapter XVi 

ANALYSIS AND INTERPRETATIO^F 

Mrs. Ji'frt II a. /'r .v.s:/ iK!( }i 
Wan! /'*. Pi^rfi r 



"All 1)1* us must woik with figures and JunubcM's 
every day. The simplest assertion in conversatior; 
. . . deals with ciuantities . . . (Statistical analysis) is 
one way of bringing big numbers down to a si'- 
where they ean be grasped with ease."- 

What we do with figures once they are colloctc • 
is the key to the usefulness of research findings. 
Without analysis and interpretation, figures have 
little or no meaning. Some degree of statis^icrd 
analysis is within the reach of * very e.vlension 
worker. 

Perhaps a definition of the terms analysis and 
interpretation will be helpful. We can think oi 
analysis us the ordering of data in such a nvanncM* 
that they yield answers to (juestions. This mean:-* 
we take something from our mass ot data and 
fuganize it so we can take hold of it. catch on to 
what it is trying to tell us, understand it. Wc break 
the whole of our inforn ation down into components 
which become meaningful to us. In layman tcrnis . 
analysis is putting data through a inill; we take our 
facts, run them through some soil of statistical pi-o- 
^'essing, and ccjme up with answers. We put in 
information and take out intelligence, 

interpretation, on the other hand, is the process 
of searching out the meanings of our answers, of 
our new intelligence. Tb's process occurs when wc 
get around to answering tlie "So what?" c|uestions. 
Interpretation leads to the establi liment of explan- 
atory concepts. 

hi the normal course of events we would e pect 
our interpretations to for uir analybi:;. It is 
essential that we have digc (analyzed) data, or 
knowledge, before we can ssign meanings and 
implications li^e.. make mt^rpl-etations). Mowcvei-. 
it is (|uite possible that our original interpretation;, 
may suggest further needs for analysis of oui* data. 
This may be the occasion for a new cycle of soi ting, 
analyzing, and interpreting. 

When are plans made for tlie analysis and inter- 
pretation of data? This is an area in which much 
help is needed. A plan for analysis needs to h 
well thought out by the investigator when a re- 
search project is developed. Instruments for col- 
lecting data must provide ihn information need<}d 



for- pin-poses of a' 
can extr£>ct ^rom 
This is :.<){ U; 
final and co.ijpjelv 



.\o amount of analysis 
Jors that are not present 
.^ever. that we can make 
for aralvsis at the be^^in- 



iung J rc: inarch j;roject. 7'he :\ oarcher must 
li vaj-'S look for ne\v concepts as observations are 
!uade. P.owever, analysis nvir;^ pvecede interpreta- 
tion, even though there ;s some overlapping. 

It is helpful lo write oi.t specific plans and to 
prepi>rr work tabk . in advance. This frequently 
renfii'-e a ^vqji deai cf tinc; on the part of the 
researcher. It is in.fMr^ant that administrators 
..nderstand ilvat ti;r:0 nd other resources must be 
provided for research. In this connection, a calen- 
dar of work, outlining t'le jobs to be done, the Mrne 
required^ and responsibilities, is helpful. 

'l>Kij?:-ocosseo of analysis and interpretation, e^ cn 
iliough of fundame-dal importance in the resi.ir'... 
process, are oiton inadenuately used by extens;;..fi 
workers. T'm:. is freqr ;Mtly ciue a limited under- 
standing' the >vasic pnnoiples involved Consrd- 
er..i)Oi.^ \vill now be ^iven to some guiding prin- 
ciples thai may bo helpful to tue inexperienced 
rese;:rc;:^r. 

Techniques of An^slysis 

Statistical analys's can be ( uite involved How- 
ever, all extmj'on w^orkers are capable of using 
S!iiiple mather,fai.c*al tec:nn.iques iivdi ^vil| serve the 
purpose for mo^t extens'cn studies. These simple 
techniques inokk.e (I) numerical counts, (2) per- 
centages, (3) 'measures of central tendency or aver- 
ages. f4) measures of dispersion, and (5) /anking. 

Numerica! Counts — For many of our purposes, 
it may be sufficient to know how.many. IIow many 
of the respondents knew the agent by name (51 did, 
17 did not, 2 did not answer)? How mar.y of the 
respondents ^vere in ea( !t age group (5 were under 
21 years. 37 were 21 through 54 years, 12 were 55 
years or over)? IIow many of those who were 
shown how to take a soil sample actually took one 
on their own land (37 expose<.l, 13 took sample, 24 
did not take . ;unpleV? 

We need sii h counts to serve as a base for what- 
ever a!i;nvsis wc do. Sometimes this is all \ve car- 
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to know: at other times this is moi-cly the: -itarting 
point lor i\u involved series of nianipuhUiuiis. Th^* 
<|ua]ily of «iiir cUita, as well as other faetors. wilW 
(lett'nnine how inucli more we are ji; Mfieci in doing 
with them by way of analysis. 

Caution should be exercised in restiieting ealcu- 
lations simple numeiical counts in some cases, 
other calcuhitions are frecpientty helpful in avoid- 
in.u oversimplification or in pronioting larification. 
This will become more evident in the discussions 
to follow. 

Percentages — Probably the next most l'rec|uent!y 
used statistic aft< the count is the percent, or the 
proportion the part is of the wkole. It is a gotnl aid 
in making relatio'isr^ps and c<Mnparisons meaning- 
ful. For example: A relationship like 67 out of 179. 
compared with out of 70. is more difficult to 
gras|^ tlian w percvnt comparedVith 43 percent. 

Another valid reason for usmg ])ercentages or 
ratios is that they ai e more easily understood. This 
is particularly true where lajge numbers arc in- 
volved. For example. 2i30 out ^d' 780 does not inean 
to(i much to '^^ost peorile; but .32 percent is mean- 
ingful. 

Although the computation of pei'centnges ■ -r*.-; . 
^lo be a^niple ant J l alhei* obvious process, tlir ; e 
several opf)ortunilies for the researcher ^^o 
wrong. 

1 Use of the Correct Base — The base (the de- 
nonnnator or divisor in malhomatical tei-ms) i.s the 
number irom which the percentage is calculated. 
.Selecting the correct base is impoj'tant. It is ecpial- 
ly important to indicate for you' readers or listen- 
ers what your base is. In a typical evaluation (pies- 
tion. tile base could be: (1) the total number^ of 
I •■s[)ondents jn the study: (2) ti:^ total number wlu) 
aiisweied the qu(vstion: or* i3i the total number to 
whom the question applied 

^ In using the tottd number of cases as a base, 
I'oneous conclusions may sometimes be drawn. 
'!'his is par^ij.ularly true if the proportion of "no 
responses** is high. Let us say that out of 100 
records, 10 did not respond to a certain Cjuestion. 
70 answered "ye.s" and 20 answei-ed *'no." Using 
the total of 100 a:^ a base, this means that 70 percent 
answered "yes.** 

'ng thr ^ame figures as above, and taking the 
numoer who answered the (|uestion at all (90) as a 
base, we find that 77 percent of those who answer- 
ed ihe Cjuestion repor ted **yes." 

The difference between these two calculations is 
7 pejcent. Which is the better answer? We do 
nof know whethei- the "no responses'' would have 
been "yes * m "no." Conseruientiy. it is essential 
tliat the analysis show either that 10 percent did 
not answer, or that the 10 ' no answer^" wen 
emitted. 

There are many situations in which a question is 
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not applicable to a respondent. Only the number 
of p(;^i sons to whom the particujar question applies 
can be used as the base for calculating a pcrcenta- 
age. C.ic example of this is in Table 

Tabic I. Broilei- practices adopted by 4-H Club 
members in X county (Number mem- 
bers raising broilers— 75} 



Practices adopted Mumber Percent 

V^Mendjers attc ndiffg meeting 



(where infoi'matinn was given) 


50 


100 


'^X^owing 1 .sriuai-c loot 
flo?H:j^pace per bird 


30 


60 


Using thermostatic controls 


20 


40 


Keeping 95 temperature for 
first week 


40 


"0 


Prole.stiiig brooders 24-48 
hours p;"ior to use 


45 


no 



This table involves responses to questions asked 
to determine whether or not 4-H Club members in 
a broiler project were using recommended prac- 
tices. In this instance, the county agent is using, as 
the base, only Ihose who received information on 
practices. Thi:. ,)rinciple is often applied when an 
attempt is being made to determine teaching effec- 
tiveness. 

Another example is the situation where a sched- 
ule to be used with a cross-sectjon of farmers has 
a (piestion on spraying orchards. Obviously, only 
those farmers' in the sample who have ' rchards will 
bo considered as the ba.sc for calculating percent- 
ages. 

:l Adding Percentages — Before we rm add per- 
centages, ve must be sure that cr . gories are 
mutually exclusive (do not overlap). For example, 
let us say we secured the following percentages' 
when re^jpondcnts indicated how they received 
i^iformation about a particular i)raciice: 



Source Percent 

-iiadio IP, 

Television 12 

Newspaper 28 

Neighbors 42 

Extension Agent 38 

Not known 1:"5 



Kven if the percentages had failed to add up to 
more than 100, we should have no difficulty in 
rc^alizing these are not exclusive categories. It is 
I)ossible that some respondents could have. gained 
inform }1 ion from more than one source, conceiv- 
ably fi-om all of the sources mentioned. ^' 
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However, many questions are such that exclusive 
categoric: are possible. This is common among 
"face" data or background information items. For 
example, th^^ -istribujion of replies to a question 
on age might oe shown as follows: 



Age group 


Number 


Percent 


Under 30 years 


53. 


9 


.?0 through 50 years 


287 


47 


over ."'^ years 


266 


44 


Total 


eo6 


"100 



In thi*' case we have selected our age groups so 
th thcM C is no overk )ping TrmUwe are justifiod 
ill adding up our percentages. 

3. Averaging Percentages — Sometimes the inex- 
perienced researcher falls iny)-1Jicerror\of report- 
ing averages of percentages^ /UtTrtrugh) this is a 
frequently observed practiJie, it is never quite 
justified. The following figures^ are the percenf- 
agcs of 4-H members who i;ojmpleted projects in 
the different districts of a State: 



Di*Btrict Number Percent 

A 31,000 74 

B 8,000 (50 

C 12,000 05 

D ■ 26,000 75 

E 11.000 50 

. F 28,000 72 

Total 116.0U0 86 (incorrect) 



the above tablo, the analyst incorrectly chose 
to report, as the tuial percent, an average of the 
six district percentages. He should have computed 
the total percentage figure by dividing the total 
number of completions (116,000) by the total State 
membership (165,000) which would result in a fig- 
ure of approximately 70 percent. Sometimes, as 
1.: che above example, the differences are not great. 
In other instances, the error can be much larger. 

^^4. Rounding Percentages — The researcher must 
deiitjcleTibw many decimal places to include when 
he reports percentages. To show too many digits 
may give a false impression of accuracy: to show 
too few may conceal the fact that differences exist. 
A safe rule of thumb to empluy is to use no more 
them are necessary to bring out the differences. 

Even when ycur data are precise enough tojustify 
refined computation of percentages, extra or un- 
needed decimals add to the difficulty of reading 
the report and they tend to make tiie figures harder 
to remember. As a general compromise, some 
writers use nearest whole percentage figures in 
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the text of their reports and, wherever necessary, 
give percentages drawn out to one or more decimal 
places in the supporting tables or i;i a footnote. 

The following thiee rules covei* the rounding of 
individual percentages: 

/' 

1. Where the dH,Mt(s) to be drop])ed repre^entCs) 
more !han half (i.e., more than .5, .50, ^ '^) the 
numerical distances between the number 
above and tiie number below, round to the 
higher number. For example, the percent- 
ages 18.6, 18.51, 17.9 — ^^when rounded to 

vhole numbc.s — become 19, 19, and 18, 
respectively. 

2. Where the digit(s^ to be dropped reprosentfs) 
less than half of the distance between two 
numbers, round to the lower number. For 
example, the percentages 18.4, 18.49, 17,3 — 
when rounded to whole numbers — become IS, 
18, and 17, rt pectively. 

3. Where the number to be rounded is exat'.ly 
haifway between two numbers, round to t^r.e 
neWe.st even number. For example, the per- 
centages 18.5, 18.50, 17.5 — when reunited 
whole numbers — all become 18. 

Jost of the time, rci ^ -r: . occurs: to the right of 
the decimal point. S..c: ocr'." to the left of 
the decimal point, ^erv.r. r*J 'x- svbsiiiulQd for 
each dropped (^\gir Ej\ ,n,^.L':S^: IOC L»'?oome3 170 
and 16,987 bee . aies -17 0?.0. ' 

Measures of Central Tendency — I'hose devices 
which help you visualize or identify the central 
chai acteristic, the representative unit, or the aver- 
; :e, are referred to collectively ns measures of 
central tendency. The ones mosv Iikeiy to be used 
by the extension researcher ar . (^j the mean, (2) 
the mode, and (3) the median. Tir : are others, but 
they require some understanding vjl advanced math- 
ematics and are not meaningful to larger por- 
tion of our probable readers or listeners. Conse- 
quently, they have limited utility in extension 
research analysis and reporting. 

The arithmetic mean, or what most people think 
of whpn they speak of average, is the most common- 
ly usrd measure. To get average income, you 
divide the toTal income reported by the number of 
people reporting mcome. To get average age, you 
divide the total number of years reported by all 
respondents by the number of respondents. To get 
average membership, you divide the total member- 
ship by the number of clubs. 

The advan^ L;e^ of usin[; the arithmetic mean 
arc: (1) it is most likely to be understood, and (2) 
it is i'Bisy to compute. Thr disadvantage is that it 
gives undue value to figure;^ at one end or the other 
of the distribution. If you were" to average club 
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jTiemberships of 5, 9, 9. 11, 13. and 37. your arith- 
metic mean would be 14, which is larger than all 
but one of the individual club memberships. 

The mode is the answer or value that most fre- 
quently occurs in a distribution For example, if 
more farmers report si^o of fa .n as 120 acres than 
any other size, then 120 is the niodel sIzl' of farm 
in the study area. 

Among its advantages are: 

1. I^^s probably what most people mean when 
they say "on the average" or ^'typical." 

2. It is the'rnost descriptive average in that it 
describes the largest number of cases in any 
one distribution. j 

3. It is not affected by extreme values. For ex- 
ample, if seven households have the following 
number of members, respectively— 1, 3, 4. 4, 

4. 8, 10 — the model size family is 4. Regardless of 
Iho number of persons in ihc smallest and largest 
household, the mode would remain the same. 

Its nuijor disadvantage is that it is significant 
only when a large number of values is available. 

•Another disadvantage is that occasions may 
arise when no one group will have numerical super- 
iority over anotlier. Quite often you come up with 
two groups the sam size, or nearly so, have 
what is called a b'unodal distribution. When this 
occurs, your analysis will be nioro difficult, but a 
atatfstician c;ui tell you how to deal with such 
situations. 

The median is the middle vahio, where half of 
the cases fall bciow and half above. To get this 
value, you arrange uirrav) your data from one ex- 
treme to the other, then proceed to count half 
way through the list of nun^bers to identify your 
m'edian valu6. In our first example, the array 
would read: 5, 9, 9. 11, 13, 37. The n'lmber half 
way^ through this array would be our median num- 
ber.' In our case, it would bo an imaginary ;iumber 
as we have an even number of answei's. \\ o can 
resolve this difficulty by saying our median number 
lies tfdtween 9 and 11, or we can split the difference 
between these nmibers ; ad call it 10. There is a 
fornuila for determining this number precisely, but 
seldom will we need to be this precise^ 

Advantages of the n^^^dian are: il \ it is fairly easy 
to calr-Mate, even w'hi^n you hav(j to compute the 
exact I edian number; and (2) i( not affected by 
extereme values or the range of data. Its disadvan- 
t:iges lie in the fact that. M) it is not as familiar as 
the ]nean to the layman. an<! (2) making up the array 
can take quite a lot of time. 

Measures of Dispersion — The i . ciuency distribu- 
tion is a classification of the answers or values into 
various group.s or classes, arranged in order of size 
or magnitude. 



Classifyirfg people according to place of residence 
as urban, rural farm, and rural nonfarm is an ex- 
ample of a frequency distribution. Such a- distri- 
bution might be as follows: 

Pla^e of residence Number Percent 



Urban 

Rural nonfarm 
Rural farm 



50 
75 

^75 



10 
15 
75 




^' many cases', measures of central tendency or * 
averages are inadequate for telling the complete 
story. This is particularly true where, the spread 
or range in the data is great and wherg extreme 
values distort the. picture. In these instances, the 
frequency distribution of replies — whether on a 
numerical or a percentage basis — can be a very 
useful technique of analysis. For example, it may 
be desirable to know the proportion of young farm 
families in a county in order to plan programs to 
meet their needs. 

The u.se of frequency distributions can be helpful 
in checking the representativeness of a f imple 
against census dota. If this is to be done, he dass 
intervals or classiqcations sljould be iden:^ .:^; ^vith 
those used'for comparison. 

In preparing a frequency distribution, cr- must 
be taken to select classes or groups (class i .L^rvals) 
thai are meaningful in terms of the' local situation. 
Another requirement is that class intervals should 
not overlap. In this connection, it would be correct 
to use "under 10/' ''lO-lO/' '^20-29,^' ore. rather . 
than "0-10," "10-30;' "^0-30/^ etc. 

There -iro, of course, more complex and precise 
ineasurements of dispersion than the techniques we 
have discussed. For the majority of our extension 
studies, however, nunierical and percentage distri- 
bution will probably suffice. Whenever we do a 
study in which our data have been collected in such 
a manner that more precise analyses are justified, 
wp, can consult with a trained statistician who will 
lell us how to use the more advanced methods. 

Ranking — Most j)eopIe liko to know where they 
stand in relation to their fellows. Clubs liK * to 
know how they compare with other clubs on num- 
ber of members, attendance and participation rates, 
and other performance ^factors. County residents 
like to know where they rank amonrj counties in the 
State on population, income, percent of farm resi- 
dents, acres in the differer:t cror s. birth and other 
vital statistics rates, and so on. 

Quite often we can use ranking.^; very effectively 
in our evaluation tasks. ^Eveii if the differences 
may be slight, knowledge that you rank well up on 
a list can be a source of satisfaction. However, if 
0 much emphasis is placed on the ranking^^ther 
than on potential achievements, it can have an 



inhibiting offisct on motivation and incdhtivc to 
improve. 

Rankings are meaningful to most folks, although 
there is a persistent tendency to interpiat rankings 
as measurements in^^tead of evidetices of sequence. 
It is easy to forget there may only a small frac- 
tion of a unit difference between' the 30th and 31st 
ranked items and a difference of several units be- 
tween the first and secon \ ranked items, or between 
the last and next to last items. 

Rankings can be made up, quite easily if there 
are only a few items to rank. ' You just make ail 
array of the items from greatest to smallest in 
terms of the units of measurements (inches, pounds, 
acres, members, completions, adoptidhs). Then you 
assign consocutivf numbers to the items accordirrg^ 
to their positions in the array. You can assign the 
' numbers from most to least or from lerist to most, 
depending on intended use. This po' :nakes no 
real difference so long as you indicate which way 
you have chosen to assign the rankings. 

•Sometimes there is difficulty if several items 
have the same size (several 5-member families, 
many 40-acre farm>, sovcral members with the 
same number of project completions, and so on). 
Frequently, in such cases, it is sufficient In assign 
the same ranking to each similar entry. 

If you are ranking averages, percentages, or 
index numbers of some kind, you may be able to 
get differences by carrying out your divisions an 
extra place or more. This is a situation where dif- 
ferences will be trifling and the resulting rankings 
may have I^.tlo meaning. 

If there is no way to break the tie, or if you feel 
there is no justification for a.ssigning different rank- 
ings, you give nil equal items the same ranking, 
Then you must be on the aiert to avoid a very corv 
nion error in ranking. You must leave places in 
your ranking for each tied entry and assign the next 
entry the first plac<j beyond those. 

If your array reads, G-9, D-8, PI-8, B-6, E-5, 
C-3, F*3, you vvo»-M '"^nort rankings, G first: A, D 
and H tied • s^ B fifth; E sixth; and C and F 

tied for seventh; not, G first; A, D and H tied for 
second; B third, E fourth; and C and F rod for fifth. 

Rankings are not used a gre?rt do^ \ in statistical 
analysis beyond simply reporting thorn, although it 
is possible to use them to make rough correlation 
tests. Because of some problems in dealing wit' 
howC'/er,Jt is best to ask a statistician to ex- 
plain the formula and the limitations o£ the statistic 
to you. 

Interpretation Techn5q\*i:»s 

After we haye collected, tabulated and analyzed 
our data through the various processes available lo 
us, we still have the task of interpreting the find- 
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ings. It is not enough merely to cite our findings 
'in descriptive fashion. We must ask ourselves the 
question, '*What do these data really mean?" More 
than that, we have to come up with the answer 
the question, or nt 'east a statement of what we 
believe the findings imply. To guide us in working 
out such a statement, there are several fundamental 
principles ,hich we will now consider. 

Genera h'zation— Most research motivated by a 
desire to make inclusive state uients based on n 
limited number of observations^ We desire ob- 
serve results in a limited setting and generalize our 
findings to a larger group or population. The 
demands such a desire places on our methodology 
has been discussed at considerable length in other 
chapters. 

We must exercis caution in drawing conclusions 
for a larger or fi^ifferent population than the one 
from whi \ the sample was drawn. If we collect 
informati >ii from only farm residents in the com- 
munity, we cannot generalize to the total population 
of the community. In fact, if the percentage of 
returns is low- (as in many mail questionnaire 
studies), or if the refusal rate is high, we may be 
justifies, ill a'pplying the findings only to the actual 
group of respondents, rathcB than to the whole of 
the sample and to the population/ from which the 
sample was drawn, as we had intended. 

Definition — We have no right to generalize for 
areas or situations where the definitions we have 
used in our study do not apply. If, we would make 
comparisons, in terms of concepts or standards or 
characteristics that are not uniformly defined, we 
must attempt to either adjust our findings so they 
will be con^jMrable, or acknowledge that compari- 
sons cannotDe made, or that they are made with 
reservations and qualifications. 

We may want to compare our findings regarding 
^ income in home demonstr. lion clubs with census 
data. Unless our def initio.: of income is the same 
as that of the census, or imless we can go back to 
the original data and work out comparable units, 
we are not justified in making income comparisons. 
If the comparison is to be valid, incomes must be 
defined in the same way. 

Objectivity — Again we must remind ourselves to 
'keep personal judgments and other evidences of 
subjectivity out of our interpretations. Now is the 
time when the attention we gave to assuring objec- 
tivity in setting up our record forms will pay off. 
/ For example, *'Did you eat an adequate brpakfast 
this morning?'* leaves the interpretati^juor **ade- 
quate*' to the person who answewK'^e question. 
*'What did you eat for breakfast?*^equires the list- 
ing of all foods eaten. The ade^acy of the break- 
fast can then be determined in line with recom- 
rii^'nded nutritional, standards. 
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Cause and Effect Relationships — -One of the 
bask- pui po.scs ol* scientific research is to c stablish 
causality dhe reason why). Now we are all aware 
of how difficult this is to acconipli.sh in the behavi- 
oral sciences. However, our scientific in [Uisitive- 
rie.ss compels us to try to do it. and wo feel certain 
that as our technicjues Improve ue will become 
inrreasin.qly successful in doing it. The best safe- 
«^uaid can omj)loy before liiaking claims for 
cause and effect relationships is to screen out all 
other likely causal factor's before drawing our con- 
clusions. 

In the table that follows wc have what looks like 
a ,i;ood case foi' the value of extension teaching. 
'I'his is an illustration from an extension s^udy de- 
sijLiiied to learn the effect of participation in Exten- 
sion on- the rate of adoption of reconnnended 
orchar'd practices. Strictly on a basis of percdntaf^e 
comparisons between ))ai'ticipators and nonparties- 
pators. we appear to have confirmation of the 
effectiveness of participation in p]xt; nsioii as a 
causal factor in the adoption of recomi ;.Muied 
oichard pi'actices. 

Table If. FariTi pi'actices cai'ried out. with relation ' 
to farm operator's' ])articipation in Ex- 
tension. 

Extervt_of Participation 

Practices adopted'. 

Active iiv^ctive None 



Porcentajjc of those with 




— percent — 




orchards who: 








Carried out 3 or more 








recommended practices 


7(5 


(58 


50 


Ap|)liod fertilizer 


63 


7B 


36 


Cultivated adequately 


7;i 


. BO 


82 


Followed spray schedu!f?~^ 


30 


20 




Tieated for borers I 


32 


20 


18 


I'l'uued trees , / 


(58 


53 


36 


Thinned fruit - ' j 




23 


20 


■ All i"-actices wercTriyfiruxT 



However, there were some (5thei* differences 
among the farmers in the sample, in addition; to 
extent of paj-ticipation. We found that the more 
active participants tended to have hi^^her education 
levels, higher income levels, and larger farm opera- 
tions. Even though more refined statistical calcu- 
lations might reveal a positivo relationship, there is 
I'oom to challenge our unquaiiiied acceptance of 
extension teaching as the sole, or at least major, 
causal factor. It now appears obvious that some 
of these other factors may be contributing factors 
in the adoption of recommended orchard practices. 

Our point here is that we must be extremely 
careful whenever we assign cause and effect rela- 
tionships. Befoi'o we make such claijns we must 
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be sure to check our relationships against all avail- 
able evidence. This means di'awing or, i>ar "face'' 
data items to make aTl likely cross-tabulations to 
identify any, associations that may supplement pur 
basic relationship in any w^iy. Tin's is also a final 
plea to include adequate **face" data -and control 
items in all data-collecting forms. ^ """^^^ 

Program Objectives — Many of our studies are 
made to determine program effectiveness.' Data 
collected for this puj^pose might be interpreted to 
point out how well program objectives" are being 
met. The.^e objectives, as suggested elsewhere, aVe 
expressed n;ost aptly in terms of specific behavior 
changes. 

In the table that follows, the desired behavior 
"hange is constant feeding of phenothiazine and 
salt. You will note that close to three out of every 
ten of the farms made the desired behavior change-, 
almost a h .If partially adopted the practice, and 
about i; fourth failed to act in the desired manner. 
The effectivenes s of the teaching will be judged in 
accordaiice with what is considered a satisfactory 
rate of change for the particular objective. 



Table III. U.^c^of phenothiazine and salt on farms 
in sample, X county, 1950. 



Degree of use 


Number of 


Percent of 


farms* 


farms* 


Feeding constantly 


35 


29 


Feeding occasionally 


55 


47 


.\ot feedijig at all 


28 


24 


Total 


118 


100 



Sollifi^i sheep and hiinbs. 



Potential — Much of our interpretation is con- 
cerned whu determining how successful we have 
b^en in reaching our potential clientele. Potentials 
are closely associated with program objectives and 
with' careful definitions of our population in terms 
of them. In 4-II Club work, for instance, the objec- 
tive may be to enroll boys and girls who are 
students in rural schools. In this case, the potential 
would be all the boys and girls of 4-11 Club age who 
are enrolled in rural schools of the area under 
consideration. 

When it comes to interpreting enrollment figures 
on a percentage basis, only those boys and girls of 
the pi'oper age and going to i ural schools would be 
considered. 

On the other hand, if all boys and girls of 4-H 
Club age w'ho live in the county ai*e encouraged to 
cnroU in 4-H Club work, then the total number of 
such young people in the county would be the 
potential clientele for the r;clion program. This 
number would serve as the base for computing thq 
percentage of the potential reached. 
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Another exami/^ of interpretation in terms of 
potentials relates to pasture improvement. One 
objective of such a program might be the syste- 
matic application of lime. In this instance, our 
potential audience is limited to* those farms having 
pasture land. Secondly, we' would consider only 
ihose pastures with soil conditions requiring lime. 
In other words, we would not have a true picture 
of the effectiveness of our teaching if we included 
all pasture acres, and less so if we included all 
acres in farms. 

T? ."n^Js — We do have a good deal of trend analy 
sis interpretation in extension^ evaluation work 
Before, during and af^r data are often the'vbasia 
of our evaluations of extension teaching 'if tectivc;> 
nes.,. . Any time we do a benchmark study. wo are 
setting, the s^age for a trend study. Trends are 
very helpful in any study of change, whether as 
indicators of progress or direction of movement. 

We\^have to be very careful when interpreting 
changes, however/as there are some factors which 
could be involved besides ihe ones under considora- 
^ tion. Frequently, what appears to be a mailed 
change may be accounted for 'almost entirely \iy 
a shift in population. . . • 

For instance, we have the example of the report 
by presumably qualified researchers stating that 
there had beeo a 10 percent inciease in milk con- 
sumption in their State during a specified period, 
something liku a shift from 2,000,000 pounds to 
2, 200, Of '0 pounds i er day in terms of whole milk 
equivalent. Actually, a realistic interpretation of 
their findings wouJd have indicated virtuaJly no 
change in milk cons^t^mption per capita during the 
period. The populatioit^ii^reased from something 
like 2,000,000 to about 2,2DO,000 persons during 
. the same period; so we can^xsee that instead of 
increasing, per capita consumption remained almost 
^ constant at approximately one ^ound per day. 

We are on even morcf dangerous ground when 
we try to intei pret trend data in terms of changes 



in dollar income over a period of yearjs. Inflation- 
ary ndencies in tl\; economy as a whole may 
account for rr^- ^anQverA increase in individual 
incomes. ^ 

When yoi» -elf involved in such inter- 

pretations . with a trained researcher 

or"a spb^ciais .itics I2 in order. 

-Summary 

After sojne preliminary comments to identify 
analysis as the ordering of data to g t answers to 
questions:, and in pretation as the process of 
sea/ching out meanings of answers, this -chapter 
points out some principles and techniques of 
analysis and interpretation. 

In our discussion of analysis techniques we refer 
to counts, jercentages, me^s^^res of central ten- 
dency and dispersion, and ranking.' Those tech^ 
' niques requiring only an elementary knowledge of 
mgthematics are emphasized. We also point out 
somd common errors in the use of oach technique. 

In ^iiscussing interpretation techniques, some' 
safeguards are suggested to assure objectivity and 
appropriateness in our appraisals. We mention 
some factors limiting how far we can generalize. 
We stress the importance of agreement on defini- 
tions when making comparisons! We point out the 
need for adequate checking before postulating 
cause and r'^'ect relationships. Finally, we discuss 
the importance and value of interpretations in the 
light of potenlisi.s, and trends. 
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Chapter XVII 

PREPARING A RESEARCH OR 
STUDY REPORT 

U'fird r. Porter 



If research findings are to be used, they niust be 
organized and made available in understandabie 
and appropriate forms. In some case.s a complete 
vvritten. report is not required. If the results of a 
study are to be used only by the investigator, or a 
few people, there may be no need for a report other 
than analysis tables. E::amples of this may include 
a summary of personnel ratings or a detei rnination 
of rnterests^f a specific group with reference to 
CO aferencqy seminars and related activities. 

In this CcKe there juiay be no distribution of writ- 
ten materials. In general, however, if research 
results are to be used by niany people, a written 
report will be necessary. ^Sometimes when the 
study is an important one, a single report will not 
do justice to the findings. In ^his case, a series of 
reports, aimed at different audiences, may need to 
.be prepared. The principles of presenting findings 
w^ould apply almost equally to both the oral and 
written presentations. 

, Data arc collected to answer questions. When 
interpreted, the;,' .supply answers to those questions. 
The report states the questions, outlines the method 
of collection and analysis of the data, and explains 
the answers obtained within the limitations of the 
study ')bjectives and finding^ 

In preparing study reports,' there are many fact- 
ors to be considered. This chapter is concerned 
with some of the more important considerations. 
Among these, we will discuss the anticipated aud- 
ience, report outline, contents and organization, 
writing styles, and methods of presenting study 
data. In addition to these, one would also need to 
consider available resources, pertinent to preparing 
and publishing repbrts, . These aspects of rGt)ort 
writing Will be discussed under the three major 
hoaUiiigs of planning, style of writing, and ways of 
presenting data. \^ 

PLANNING A STUDY REPORT 



The Audience 

Any writer who hopes to do an C 
should have some idea of his prob 
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ttive report 
le audience. 



Some understanding of its needs, interests and 
background will help him decide which points to 
stress in his^presentation. With this under-^.tanding, 
he will be in a better position to know when to slow 
down and explain thin^^s; he will know which details 
to mention briefly and which ones to spell out; he 
will know the proper degree of familiarity to use 
in his writing style. 

For our present purposes, we may conceive of 
three types of audiences (readers or users) for ex- 
tension study reports: (H^other research people; (2) 
all professional extension \vorJiers"and other educa- 
tional per.-onneI in related fields; (3) the lay aud- 
ience. In any one of these categories there may be 
several subclasses. For example, the lay audience 
as thought of in this chapter may consist of rural, 
urban, farm, nonfarm or other people not especially 
concerned with professional extension work. 

If a report is prepared for other research people, 
it is important to include the following: 

1., ^Tahiti of contents. 

2- Summary statement of findings. 

3, Specific purposes and study objectives. 

4, Significance of the study. This should include 
reasons for undertaking the project and its 
mjCaning to the audience concernecfr"^ 

^ P)rocedures used in the study. This might 
involve a complete discussion of: population 
and sampling; methods of data collection — 
nSail, personal interview, telephone, r com-,, 
binations of these methods, recortHm-ms used; 
selection and training of interview^; the 
dates on which field data were coHectea\and 
rate of responses. \ 

6. Stai tical methods and techniques used in' 
the tabulation, analysis and interpretation of 
the data. 

7. Results obtained in relation to the objectives 
of the sl}lcly as well as other findirfgs^f ' a 

^^-^v '"b^roduct" luitdic ^^^^ J 

8. Implications of the findingsjii^elation to tho 
purpose^: of the specific ;^tudy or research, pro- 
ject. This may hv iCSs^T:i^rtant for research 
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persons than for the other audiences previous- 
ly :nontioned. - . 

9. Iilehtincation, including the names of people 
responsible for planning and conducting tbu. 
study, date and place of publication, acknowl- 
edgments. 

10. Reproduction d'f survey schedules or ques- 
tionna^es. • 

'I'hc order in which these phases of the report 
are presented is sometimes determined by the 
author. In other instances, the institution or agency 
sponsoring the study makes the decision. 

If the audience is expected to be professional 
extension and other ieducational leaders, certain 
adaptations of the above content, organization and 
formal of the report may need to be made. For 
this group, details as to statistical procedures, 
sampling techniques and other methods used in 
conducting the study may be omitted' or at least 
discussed only briefly. It is important to report the 
findings in such a way that the extension worker 
can see their application to his own program." This 
may be the improvement of a method, planning 
procedures, or program content. 

For the lay audience, a research report needs 
to be concise, non-technical, easy to read, and may 
contain only major findings from the study. o Rais- 
ing questions relating to the significance of the 
research for the ^jarticular audience may make a 
report more personal, appealing, and, therefore, 
meaningful. 

The above discussion implies that there may be 
more than one report prepared to meet the needs 
of various audiences. In some cases there may he 
several reports. For example, there may be a very 
complete, detailed an ' highly technical report for 
the audience of research personnel; a report for 
professional extension workers containing the ma- 
jor findings and implications, with loss emphasis on 
research procedures and technical details; and, 
finally, a relatively simple and brief report for lay 
people highlighting the major findings in a popular 
style. 

It is also appropriate to think of a highly compre- 
hensive study being reported in several specialized 
''editions." For example, a survey may show cer- 
tain data on topics of interest to different special- 
ists. 



Resources 



Sorr^e other factors to be considered in planning 
a research report include r- M ces available to the 
researcher and the institu.iuii or agency involved. 
Among these are time, finances, facilities and per- 
sonnel. 

Time is of importance from at least two staiuU 



points; that of time availablOj^to prepare the report, 
and the urgency in terms ol timeliness and use of 
the findings. ^ Adeciuate time should be made avail- 
ab!f^ lor the preparation of the research report. It 
i.s wasteful of time and money to make an investiga- 
tion without, adecjuateiy reporting the results. 

The money, .facilities and skills available for writ- 
ing and illustrating will influence the type, number 
and scope of K'search reports. This should also 
be taken into.aecount when the research projdct or 
evaluation is planned; 

It is suggested that the author of a report work 
very closely with the editorial staff in charge of 
publications, both in planning and writing a report. 
Editors can be of invaluable help in selection of 
visuals, improving readability, and effective pre- 
sentation. 



WRITING A STUDY REPORT 

The Report Outline 

**In planning the report, try to think through the 
story that you want to tell your readers; then write 
that story in your own words. - Report writing is 
not top different from other writing.''- Effective 
writing is a tool that helps toMnsure understanding 
and use of study findings. 

Most writers who contemplate writing a report 
find it helpful to have some sort of ouiline to work, 
with. It will assure order in the finished work and 
it will help hold down repetition and guard against 
omissions. Such an outline need not be spelled out 
in complete detail and does not always have to be 
writtera, but the various points to be included should 
h' rtiven careful thought before Victual writing is 
started.. ' 

Organization and Content 

Select only the facts that are important and sig- 
nificant for Wi various audiences for whom the 
report is being written. It is helpful to arrange 
facts in logical order so thac they may be easily 
followed by tho v ef:i<kr. Tho use of subheads helps 
to 9all attention to points lliat need to be emphasiz- 
ed and facilitates readability. 

Findings should be reported in terms of the study 
ob^tives. Wherever results are not conclusivre, 
sorr^b explanation should be made. It is also imp^i^ 
laiit that an objective approach be maintained in 
interpreting the findings. Where the writer is 
expressing. a/ aiue judgment, this should be made ^ 
apparent For example, a writer may state that the 
rate of adoption of a practice was "high*' when 25 
percent of the respondents reported use of the 
practice. In reality 25 percent may or may not be 
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a "hii^h" rate, depenciing on the circiimslanccs or 
previou.sly established standards. 

There are several important considerations re- 
f^ardin^' the organization and content of a study 
report. 

L Title — All reports should start off u it!: a title, 
-either on a separate page or at the h^;;d c the^ 
ai ticle. Ui- the more •'po,':'uku- types of repoi : it 
should be short and designed to attract attention 
yet give some clue to contents; in the more formal r 
imports it can be longer and should givv a more 
positive clue to contents. 

Most of the time the author's name appeaiyS im- 
mediately afte.r the title or somewhere on thd title 
I)age. Sometimes .his name does not appear j)n the 
more popular types of report and his agejicy 
designated as the author. 

2. Summary Statement of Study — An increasing- 
ly popular practice in reofMl writing is to p'ace a 
summary of the report in the fore part of the pub- 
lication. It may come before or after the table of 

'contents-and list'of charts, figures and tables, but 
it always precedes the body of tlie report. It is not 
designed to take the place of the chaptei* on con- 
clusions which continues to be i)lnced at th'e end. 
Rather, it pulls togethei* the highlights of the entire 
report and jSresents them so that, (1) the mildly 
interested reader can get the main idea without 
reading further, and {2) the undecided reader can 
get enough infoH^iation to decide whether or not 
to j;ead the complete report. 

3. Table of Contents — Quite often the table of 
contents is omitted from short reports, and those' 
of intermediate length which are written for the 
lay audience. Any of the longer reports, regardless 
of 'the intended audience, need a tabic of coiitonts 
to^speed up reference or to locate chapters or sec- 
tions in which the i-eader may be most interested. 
In the very detailed re])oi*t. the table uf contents 
may list .subtitles as well as chapter and section 
titles. 

4. Study Objectives and Procedures — Any re- 

l)ort. whether it is a professional monograph, a 
graduate student thesis, a journal ai'ticle or note, 
a miscellaneous publication, or a leaflet, requires 
an introductory section of some sort. 

One' of the items to be covered in such a section 
is a .statement of the objectives or purposc^s of the 
study. J'his statement should also contain the rea- 
son^, why the objectives were considei-ed important, 
and the probable use that could be made of the 
findings of the study. 

Somewhere in the introductory section the 
should be some comments on how the study wa.. 
made. The amount of space to devote to this will ' 
depend in large part o -he intended audience. Wc 



would ]iot expect certain lay people to be interested 
in all of the details, althdugh it is a good idea to 
give them some notion of where the study was 
made, w^io was involved, and'-^procedures used irl 
collecting the information. 

The educators would want a little more back- 
ground information to aid them in assessing the 
'merits, of the dtudy. ^ 

The details o\a procedures will be of major inter- 
est to other resem chers. They w^ill want to know^ 
- our population and the sampling plan. They will 
want to know what\kind of data collecting* instru- 
ments we used, and they would like to find a copy- 
of each in the appendix section. They will' expect 
to find some notes- identifying the research team 
and- what training or conditioning was provided 
for them. They will be interested in details like 
the inclusive datps of the "field work, any special 
techniques that were used during the field work 
phase, the response rate, ancl so on. ^ 

5. Findings — When you have completed your 
introductory section, you start reporting your find- 
ings, sometimes with the help of tables, charts, 
graphs and figures, or reference to appropriate 
appendix items. 

You should report your results or findings in 
termr; of your study objectives and should present 
the evidence both for and against your hypotheses. 
When you feel you should express a person opinion. 
yon i^hould say so. Whenever results are not con- 
clu-'ive, some explanation should be made. You 
should also feel obligated to mention any fiiidings 
of a byproduct nature which you consider to be 
coniributions to knowledge or understanding any-' 
where in the general area of your study. The limi- 
tations .of the study should also be pointed out, in 
all fairness to the reader. 

6. Implications — After you have presented your 
findings, you have an obligation to make some com- 
ments as to what the findings mean. At this point 
it is extremely important to involve subject matter 
specialists in .interpreting the JFindings and in sug- 
gesting implications. Now is the occasion for sug- 
gesting related topics for future research and for 
pointing out facets of your own study which should 
be examined more closely, possibly v>ith new data. 

The need to include implications in your report 
is most urgent when you are writing for your lay 
audience. This audience is least likely to recog- 
nize the possibilities of your findings, partly be- 
cause of their less thorough knowledge of the sub- 
ject, but also because of less familiarity with deal- 
ing in cause and effect relationships. 

Most members of your educator audience will 
also need^'^'some assistance with implications. In 
addition to your implications, they \^^ilI draw their 
own after they have considered the report in rela- 



tion to J|j;icir own experiences. Because they do 
this, you have the ackhtional responsibility of being 
certain you make no implications which ygiiv data 
do not justify. 

7. Documentation — Most reports of research re- 
quire a certain amounl of substantiation from 
sources other than the research data at hand. Some- 
tunes, in. your report, you may waht to make com- 
parisons -of your results with those bbtained from 
other sources. These other sources should be iden- 
tified by footnotes or notes at the end of the chapter 
or section. All of the sources may be assenibled in 
a bibliography to accompany the report, either as 
part of or prior io the appciUlix.. 

The bibliography and footnotes frequently are 
oi^iitted from icports for the, lay audience but they 
should be in rop' its intended for educators and 
researchers. , 

8. Appendix — This section serves as V''<^ catchall 
for items which had to be omitted from the body 
of the report for one reason or another but which 
were important enough to be included with the 
report.' Sometimes side issues are elaborated on 
in separate appendix notes.. This is where you put 
your working tables which ace not needed for sup- 
porting the test but which might be of considerable 
interest to some of yaur readers. 

Here you can elaborate on your, sampling plan or 
demonstrate how to derive the statistical formulae 
you employed. You can include extra maps, charts; 
record forms used in data collecting, and any de- 
vices >oa may have worked up as a result of your 
.'^'findings. ' ' 

Although used extensively in scientific research 
reporting, appendix se,ctions are noti) common in 
popular reports.' 

Style of Writing 

Effective writing is a tool that helps assure both 
understanding and use of study findings. Mrs. 
Cowing, says thcit the specialist cannot hope to 
write '*best selJer" articles while reporting research 
findings, but he can make his report more read- 
able. She offers this advice: 

''You technical writers cannot assume that lay 
leaders^ will be interested in your technical in- 
formation. You have to identify your technical 
subject with their interests; let them know it 
will "pay them to read it. Slant your writings to 
their mterests as well as to their needs. Pack 
your writing with appeal; tell them it's "gopd-to-- 
eat" as well a-: **good-for-you.'* Camouflage what 
you think they ought to read with what they 
want to read.'''' 

J- ••• ' ' 

Although there arc occasions in extensiorj re- 
search reporting when 'professionar' v. riting slyies 




are in order, most of the' writing w^e do in Extension 
will be on the '*popular!' side. For this reason, we 
will devote our remarks to ways of improving our 
writing style to make it more appealing to our 
educators and lay audiences. 

Some characteristics of effective popular writing 
are to: (IJ.use the personal approach; (2) use short 
sentences whenever possible; '3) i/b-.. the sirr»p)es*: 
word that will carry your interfici .irJ moaniug, and 
be sure a long oj* technical word ; > understood when 
have to use it (4) confine your remarks to the main 
facts, arid keep them flowing in a logical order; (5) 
hold down the nui1;iber of qualifying statements, 
statistical references and parenthetical comments; 
(6) use the simplest illustrations or examples "that 
will accomplish your purpose; and (7) use present 
tense ..nd active vbrbs. 

Most of us object to rewriting, but tUal is the 
best way to improve the readability uf our writing 
efforts. If popular writing does not ccfme naturally 
to you, just write your report to include all the 
points you want fo make, using whatever stjjo you 
like. Then go back over it and bj-eak^p those long 
paragraphs into shorter ones containing a s?hg]e 
idea. 

Go through again and-change from the less per- 
sonal to the xnore'personal pronouns. Next, sub- 
stitute shorter words for as-mahy of the long words ■ 
as you can. When you must use technical w^oras, 
define each one the first time it appears. Take a 
second look at any sentence over three typewritten 
lines long. See if you can make it shorf.er, or re- 
write it as several short sentences. 

You may have to go back and regroup some of 
your. paragraphs and this may call for some difft-r- 
ent transition phrases or sentences. Work in a few 
short subheadings and see if any words or phrases 
in sentences need underlining for emphasis. Type 
up a clean copy, and see how easy it is to read and 
understand! 



METHODS OF PRESENTING DATA 

There are several things :n addition to writing 
style that an author must consider while preparing 
his report. None of our audiences would be pleased 
with a report consisting of narrative description 
only. They like the presentation to contain naoro 
than the printed word. We will discuss some of 
the more favored .ways of presenting research data. 

Narrative Description . 

The most common form of presentation, either 
alone or supplemented b|y other forms, is the narra- 
tive report of findings, \\accompanied by whatever 
conclusions, implications and recommendations the 
au;pior cares to make.-LThe narrative slatemcnt 



alone can become vei\\- borfng, but it is appreciated 
by those uho do not care to puzzle through tables 
or charts. 

Tables 

We use tables to bring together in rows and col- 
umns larger amounts of data than we would care 
to describe in wor^ls. W'e are still at liberty to 
make whatever C(innnents we want to about the 
(lata, but we do not have to include so much detail 
when we provide lab!o5. containing all I'clcvant data. 
Large tables, like Table I. generally are assigned to 
the appendix section; smallei' ones, like the next 
tw(j,.are nior'e apt to follow along with the nai-rative 
de\'elo|)ment of the report. 



Table I. Pereenti?ge of Respondents 14 Years Qld 
and Over ^Vho Had. Compieted a Specifi c' 
Number of Grades- in School, by Age; 
Sex. and Industry Group, Upper Monon- 
gahehr \*alley, West Virginia, .July 1954. 



Table 11. Ideas Used in Teaching Economy Meals 
to Metnbers and Nonmembers 




s. .< I ►i s. - 



"'i'he second tabh* is one designed to illustrate a 
few simple comparisons. It may not involve all 
resp{)ndents and may not exhaust the information 
provided by a single (|uestion. but it pi'osents the 
data to support the narrative at the particular point 
in the report. It recpiires a title, headings and a 
ft»w guidelines, but may not have subheadings or 
involved breakdowns in the stub. 



Subject 



Percent 
Members Nonmembers 



Know ing w hen one food is 
a better buy than another 


80 


4-9 


KnoAving what to look for 
ill iLcuiiii^ xaijei^ 


/ if 




Knowing how to make substi- 
tutions in market list ^vith- 
out changing food values 


77 


35 


.Meal planning 


71 


35 


Making a market list 


G9 


35 


Using basic seven foods chart 


68 


40 



The third table is a kind which is used cjuite often 
to point up some simp'e relationship. This kind is 
included in the text, does not carry a number or 
titlb and usually contains only one colunni of fig- 
ures. 



20'.? 
12'} 

42'? 



Reading and studying 
Sewing 

Family recreational activities 
Laundry work 

Graphs and Charts 



There are many kinds of charts, graphs, jnaps, 
sketches and diagia.nis that can be used to present 
data. Compared with tabular methods, graphs and 
charts have some advantages as well as limitations. 
They arc more likely to attract attention and im- 
press the facts upon the minds of the readers. They 
are also more limited as to the quantity of data that 
can . be presented effectively. A single chart or 
graph should not be complicated or attempt to 
l)i'esent too many facts which might be confusing. 
Unlike tables, graphic methods can usually show 
only approximate values. 

Line charts consist of broken or carved lines that 
arc used to picture a time series or trend and fre- 
quency distributions. Care must be used in select- 
ing the intervals on both the horizontal axis (base- 
line) and the vertical axis. False visual impressions 
can bo created by using intervals that are either 
too broad or too narrow for the data in question. 
Curves or line charts are not appropriate where the 
items or units repi-esented are discreet or where it 
is incorrect to read off values that lie between the 
intervals on the baseline. In such cases, a bar chart 
is to be preferred. 
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Bar charts are made up oriiorizontal orA-ertical 
bars and are used niostly I'or coniparhig inagni-/ 
tudes. The difference in lenylh of bai's on a chart 
i.s. for most people, ca.siei* to see than the difference 
between numerical figures. Bars are also used 
advantageously for showing component parts, that 
is. the parts of a total. 

Pie charts have a wide appeal and are particularly 
effective in indicating the whole and component 
parts. Greater accuracy in reading pie charts, is 
insured if numerals are written on the various parts 
of the graph. 

Pictorial charts lend variety and attract attention. 
However, they should not be used to show too many 
ideas at once. The main rule to remember is that 
an increa.se in magnitude is indicated by an increase 
in the number of characters and not by an increase 
in the size of the pictures or symbols! Symbols 
should be self-explanatory. 

Summary ' 

Research has little value if it is never pulled 
together into some form of report. Although an 
oral report may suffice for 'some occasions, we 
genei'ally think of the written report when we think 
of reporting. To guide the author in preparing the 
report, we mention several things vvhich can sim- 
plify and give direction to his efforts. _^ 

The first point to consider is the probable or in- 
tended audience. We identify thj-ee Vrfcijor classes 
of audience, the researcher, educator and lay aud- 
ience, and give some ot the core characteristics of 
each. 

The next item we co -^idcr is the outline and 
organization of the report. Its major elements are 
the title; summary statement; tables of contents; 
objectives and procedures: findings: impUcations; 
docinnentation; and appendix. 

In our discussion of style in writing the report, 
we stress the technicjues for achieving easier and 
more readable manuscripts. Most of these involve 
simplicity, brevity and organization, and are most 
efficienfly accomplished by conscientious rewriting 



with the objectives of effective jDopular writing 
clearly before us. 

Our final topic is methods of presenting data. We 
suggest the use of and briefly describe illustrative 
materiajs. such as tables, charts, maps, graphs and 
l)ictures! 

In the final analysis, decisions on writing style 
and methods of presentation must depend on the 
, intended purposes of the report and on its prospec- 
tive leaders. A report must tol! the story effec- 
tively, but this is not enough. There must be a 
personal appeal that makes the reader say to him- 
self, "This report has something in it for me/' 
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THE USES OF EVALUATION RESULTS 



Chopter XVIII 

APPLYING THE RESULTS OF 
EVALUATION IN EXTENSION 

. Joseph L. Matthars 



Four Objectives 

The objectives of this chapter are to help you: 

J. Appreciate the poter.tial for .gain that can 
conic from using l esult.s of evaluation in your 
extension \wrk. 

2. Appreciate the importance of evaluation pro- 
cedures as aids to getting results incorporated - 
into action plans. 

3. Learn ways of getting results from extension 
evaluation rnd other s^^l science research 
used. 

4. Be motivated, to use results from extension 
evaluation in your own work. 

Three Ways to Improve Practices 

We can use three ways to improve our extension 
educational practices.* The easiest wny, and the 
one least likely to provide the best solution, is to 
make changes on the basis of our subjective impres- 
sions of what the problems are. With a minimum 
of testing, or accumulation of objective evidence* 
we can make judgments about probable conse- 
quences or outcomes of our contemplated improve- 
ments, and be guided accordingly. 

Another relatively easy way is to ask a profes- 
sional research person to study the problem and 
suggest solutions. We canol be criticized for using 
this way on grounds of unreliability of results or 
defective methodology, but we can be criticized for 
passing up the learning experience of finding out 
for ourselves. At any rate, there will be few occa- 
sions when we can afford to have a professional do 
the kind of job that is needed. More than likely 
we vvould have to settle for his stock answers based 
on reasonably similar situations reported in the 
literature. 

Somewhere in the middle ground between these 
two methods lies the way which is the subject of 
this manual. Evaluation combines the advantages 
of first-hand association inherent in the first way, 
with some of the more manageable objective and 
statistically sound techniques of the latter, into a 
workable plan for use by any r easonably competent 
extension wbrkor. 



The final product resembles traditifinal research 
in most fundamental ways. In each there is an 
attempt to define the problem with precision, to 
derive the hypotheses from the best available back- 
ground information, to follow methods that will 
provide facts rather than subjective impressions, 
and to generalize from the cividence^with caution. 

Basic Differences 

The one basic difference centers on the matter of 
the motivation of persons making the study. The 
typical research person is motivated by a desire to 
arrive at *'truth.'* In his thinking, the degree of 
excellence of the work depends on the degree to 
which it contributes to. establishing generalizations 
that have wide applfcability. 

On the other hand, those of us who are involved 
in evaluating our own efforts are motivated by a 
desire to improve ouH^wn wdys of doing our work. 
When such is our objective, coming up with a/iswers 
yhat have relatively unrestricted application in Ex- 
tension are of little immediate concern. We want 
a; swers that apply to our Own particular working 
-situation. 

This kind of purpose does not mean that our 
approach to evaluation should be one that restricts 
application of our findings to a particuhir situation. 
Quite . the contrary! The sound research methods 
we apply while evaluating a particular situation 
should produce clues or suggest hypotheses that 
can, in some degree, be useful for analyzing a simi- 
lar situation in other settings, as well as our own. 
.We owe it to our fellows to da the kind of evaluation 
job that will make our findings as meaningful and 
useful as possible to them, so long as they first 
answer our own questions. 

In addition to fundamental orientation, the workt 
ing conditions under which the two types of investi- 
gators carry out their incjuiries may differ^ appre- 
ciably. Traditional researchers in education at- 
tempt to control the study setting so that many of 
the variables normally involved in a real situation 
are ruled out or controlled in some way by the 
study design. By so doing, they gain precision at 
the expense of relevance of findings. 



[89] 



ExtcMision workers who engage in evaluation must 
conduct their inquiries in the conii)licatcd climate 
of their regular activities. Because of the large 
number of variables in such a setting that cannot be 
controlled, the- resulting evaluation may be lacking 
in precision. Neverthele^, results so arrived at 
have real moaning becaifse they are derived from 
an inquiry carried out in a real situation. Any solu- 
tions suggested by such findings would not have 
to be translated from the laboratoi\v setting to the 
real one. 

Disadvantages of Calling in 
Professional Researchers 

Relying on the professional research person to 
take care of our evaluation problems has two basic 
limitations. We have already suggested the first 
one. Such a person cannot study individual prob- 
lems in any strict sense of the word because his 
orientation is toward generalization. 

* Impersonal — He may be able to think of your 
problem as an opportunity to try out a pilot study, 
in which event he can personalize the study enough 
to provide the kind of answers you need. In most 
ca.ics. however, he must design studies which pro- 
mise to contribute to general rather than specific 
knowledge. When most of our problems are pecu- 
liar to a situation and to the people who a| e involved 
in it, we cannot hope to gain maximum benefit from 
the kind of study he is pi cpared to make. 

General — A second limitation is that, even when- 
the professional researcher is able to make sound 
recommendations, we will have' difficulty incorpo- 
rating fhem into our day-to-day activities. It is 
ea.sy la talk about a better way of doing something 
as a result of reading abou^ or being told about 
what others say should be done. However, there is 
a great difference in likelihood of acceptance be- 
tween change commended by others, 'however 
qualified to make recommendations, and change 
suggested by our own careful examination of. the 
situation. 

Detached — There is a wide gap between talking 
change and accomplishing change. The surest way 
to bridge the gap is for us to participate actively in 
decision mak'ng as to what the* change should be 
so that we thoroughly understand the i7iatter and 
do not have to waste time convincing ourselves be- 
fore we can hope to convince others. Then we can 
proceed to convince others by example as wcH as 
talk. 

We can expect our associates and clientele to 
react much more readily to the suggestion and 
example of someone they know personaHy, such as 
ourselves than to the most positively uttered rec- 
O' ...lions of the professional researcher. Such 



a person cannot afford to take tlie time to identify 
])roperly with our group. He has to rely for rap- 
port on his professionaT title, the prestige of his ' 
sponsoring agency, or whatever aura of authority 
we can build up for him! "^'^ 

Getting Results 

This suggests the next point we might consider: 
'*How can we go about getting the results from 
evaluation used?'' A related question is, *'WhD can 
use the findings from evaluation?'' Obviously', the 
only ones who can use the results of evaluation are 
the persons who are involved in the situation that 
was studied or in another situation that is similar. 

The n(!?xt obvious thought is that, regardless of' 
who is invoh;ed and regardless of the situation, the 
results must be made known to the potential users.. 
The quality of the knowing is critical. Adequate 
means of involvement must be devised if the knowl- 
edge is to be used to fullest advantage. 

We use involvement here in the sense of bringing 
appropriate persons into intimate contact with 
results of the evaluaMon, Ideally, this is achieved 
most fully when the persons in a position to use the 
results actually do the evaluation. 

The problem is little "different from getting 
farmers to use the results of scientific research in- 
agricultural production. The difference, in the 
main, is in the matter of motivation and the con- 
sequences of not^ using. When the advantages are 
not readily evident, or when the penalties for non- 
compliance are not immediately apparent, it is 
much harder to convince anyone that action is 
necessary. 

Involving Others in Evaluation 

As more people become involved in an evaluation, 
the difficulty of fully involving all of them increases 
out of proportion to increase in numbers. Although 
we cannot hope to keep all of them involved com- 
pletely all of the time, we can work out ways to 
keep as many of them as possible involved as com- 
pletely as possible. We will discuss things that we 
can do to this end' in terms of five stages and the 
main questions to be answered at each. 
. We must realize that some of these suggestions 
will not apply for all evaluation efforts, but most of 
them will have quite general application. Many 
evaluation efforts are concerned- with the work of 
a single person and may be related only to a single 
part of his work. In such evalurftfonSj there may be 
little or no need to involve others. Nevertheless, 
there arc many occasions when it is best to share 
with others the 3;esults of even the simplest evalua- 
tions. P^or this reason, most of the following will 
apply in some degree to all levels of evaluation. 
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1. When -^Deciding to Evaluate — Two questions 
come to niiiul at this staye: "Shall \vc ovaluate?" 
and "Exftclly^what shall we evaluate?'' Evaluation 
should be undertaken in response to an expi-esscd 
.desire on the part of the potential u.sers of the 
results. 

Soaietinics we may have to take it upon ourselves 
to bring the need for evaluation to the attention 
of our assoeiates or clientele and to kindle a desire 
in theiu to have the evaluation take plaee. Fre- 
quently, just showing how to evaluate is enough to 
-stimulate a desire to try it. We can , do this our- 
selves or bring in someone from outside the group 
*to (lo it for us. This- can lead to interesting them 
/ in reading and reporting on findings of evaluations \ 
or evtension research -carried out elsewhere, with ^" 
the idea of deciding whether or not evaUuition is . 
^ needed locally. 

Qnce the idea of undertaking an evaluation be- 
gins receiving serious consideration, the various^ 
members can be kept involved by encouraging them ^ 
to contiMbute ideas for the evaluation itself and for 
how they might be able to make use of results in 
their own work. This can be done with each indi- 
vidual in separate conferences, in meetings o^he 
entire group, and in committee work. ,Actu>ij 
should include consideration of possrble findingf 



3. While Evaluation Is in Progress — '*How- are we 
doing?" would be .the main cjuestion asked during 
this phase Folks may not care much about going 
. out of their way ttflind out for themselves, but they 
like to have some idea of what progress is being 
niade^. Whoever has leadership responsibility for 
the evaluation must see that periodic progress 
reports are made so that interested persons can 
keep in touch, and not have the opportunity to 
forget all about the matter. — 

The difficulty of reporting satisfactorily increases 
directly with the size of the group concerned and 
inversely with the amount of involvement that can 
be maintained. The larger the group involved, the * 
more virgent will be the demands on ingenuity for 
devising ways of keeping the members informed 
and- involved. .Key subordinates will have to assume 
ever -larger^ responsibilities in this regard as the 
leaders will have to devote ever larger portions of 
meii time to coordinating and expediting activities, 
to executing the study plan and to processing the 
data., ^ . • ■ 

A reasonable number of fellow extension person- 
nel and qualified laymen can be kept intimately ^ 
involved with data collecting and processing and/ 
in some instances, with* analyzing and interpreting 
tasks. Others will have' to be kept in toucH by 



and the changes *hat might be in order as a conse- \ periodic announcements of progress so they will 
quencc- Discussion should include possible altcrna- ^^not have to be brought back ''info the fold" when 
tive courses of action and their probtible conse- 



quences, constructive or otherwise. 

2. While PJanning the Evaluation — ^^The next 
phase is focused by the question, 'iHow shall we 
do the evaluation?*' Now is the time to crystallize 
thinking about extension objectives related to the 
particular topic by research, discussion and formal 
stj^tcments of understanding. This is also an excel- 
Icnl occasion for us to remind our associates and 
clientele of related basic extension objectives and 
ideals. ^ 

This is when available alternative^ must be con- 
sidered to decide what kind and quality of evalua- 
tion to do. Thorough discussion of methodology 
and what can be expected in the way of results 
frohi the various methods will contribute to 'better 
understood decisions regarding budgetary matters. 
It will also mean better acceptance and more likely 
use of findings when the background is understood. 

The entire group can contribute ideas to be work- 
ed, into the evaluation plan and to be incoi porated 
into the statement of purpo*^os; leaders and fellow 
extension workers can consolidate the contribu- 
tions into the final plan. All can volunteer to help 
when needed v/ith data collecting and processing 
and, later, to help spread knowledge of the findings. 
A passive contribution all can make is to supply, 
without hesitation or reservation, any relevant per- 
sonal data that may be required for the evaluation. 



they can make their next big contribution by aiding 
with word-of-mouth dissemination of results of the 
evaluation. 

4. When the Evaluation Is Completed — When we 
have assembled the information and there are some 
results to report, we have reached the fourth stage- 
Now we should be ready to deal with the question, 
*'What did we find-out?" If the evj^luation is quite 
elaborate, it is a good idea to getjout tentative or 
preliminary reports on useable information as sooji 
as we can assemble it. This gives associates ap 
opportunity to get to work on the implications and 
' some likely applications before interest cools off, 
and to sustain interest until the final report is 
ready for distribution. 

They can begin work on foUowup reports and 
can prepare reports, news releases and radio or 
television talks for immediate use. They can tailor 
reports for the different groups within their aud- 
iences and can incorporate the material into their 
teaching. When the final report is ready, ,they 
should have worked up such a thonough under- 
standing of the elements of the study that they can 
devote their full attention to grasping the *'b.g 
picture" as it relates to their work and responsi- 
bilitiest 

Insofar as the immediate group is concerned, the' 
final report should be something of an anticlimax 
If members have been kept properly informed, they 
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should be aware of most of the contents of the final 
, draft. Their main "use of the document should, be 
for future reference or as an aid in' explaining the 
evaluation to others. However, some will want to 
use itAvhen documenting foUowup re'ports relating 
its findings to those of evaluations done elsewhere, 
or to suggest "areas for future evaluations. Such 
uses are, of course, in addition to the basic one of 
, presenting the facts in such a manner that deci- 
sions regarding change or direction of attion can 
be made with confidence. In this capacity its 
period of heaivy use will depend 'on reaction to 
contents. 

5. Afterward — Stage uive comes after we have 
^completed the evaluation and have made our re- 
Sports. The sort of question we must now a^k our- 
selves is, ''Who should be informed of our evalua- 
tion?" Q^ur responsibility now becomes a matter 
of sharing with others not directly concerned in the . 
original evaluation but who are^n positions where 
the informatiorL we have assembled mig.ht be used 
to advantage. 

We should always make it a practice to pass along 
our findings to the State extension training and 
researcl\ persons who will be happy to see that our 
information conies to the attention of others in the 
State, or Jn other States, who might be interested. 
If they have not alreadj*^ been involved in the evalu- 
ation, you may Want to involve them at this stage 
by securing their assistance in helping you imple- 
ment the action suggested by the evaluation. 

This is.no time to neglect the folks in the study 
area who may have been involved only incidentally 
up to this point. They should have an opportunity 
to see the final report to be assured that the e /al- 
nation was completed (that the evaluator wasn't 
just doing some busy work) and to get some idea 
of what comes next. It will help thorn gain back- 
ground for the discussion, lectures, radio and tele- 
vision talks', demonstrations, and the like thatXvill 



be coming along in the followup action, and will 
help thorn gra.sp more quickly the intent of the plan 
as a whole. ' • . ' 

Summary 

We opened this chapter with a resume of ^ the 
ti.rce conventional ways of gaining answers to ({ue.s- 
tions. (improve extension educational practices): 
Subjective impressions, consultation with pi'ofes- 
sional researchey s, and evaluation. Wc suggested 
disadvantages of (he first two which are resolved 
in the last way, . '. ^ 

Next, ^ve discussed the difference in orientation 
or motivation between the evaluator and the profes- 
sional ^'esearcher, indicating that the former needs^ 
answers to specific questions and that the latter 
must strive for generalizing answers." We also, 
suggested that the devices of the evaluator, at least 
when working on his own problems, help speed up 
if not eliminate the awareness phase, and bring the 
action phase into focus much cjuicker. 

We closed with a discussion of ways to maximize 
involvement in the evaluation as an aid to s.iK:ceiJsful 
eventual accomplishment of indicated action. We 
suggested things that can be done at each of five 
stages (while thinking about the need for evaluation, 
while preparing the evaluation, while performing 
the evaluation, when the evaluation is completed, 
and while the' indicated action is being carried out) 
to help smooth out the process. 
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Chapter XIX 

UNDERSTANDING AND USING 
EVALUApON REPORTS 

JoHcpJi L, Matthews 



Our objectives in this chapter are to encourage 
extension workers to: 

1. Appreciate evaluation* and extension reports 
as sources of u. aful knowledge.' ^ . 
. 2. Acknowledge that jjuorganized approach is 
necessary for eflTcieTrt use of evaluatioh arid 
extension research report^. 
3. Learn how to evaluate a report and how to 
» • apply the findings reported in it. 

Reports Must Communicate 

At times we may lose sight of the real purpose of 
evaluation iind research reports, -Basically/ a report 
is writtea by someone to communicate what he, 
usually the eValuator or research person, thinks is 
important for the reader to khow^bout the subject 
on which he is reporting. Although there are cer- 
tain things that the writer can do to make his com-- 
munication more palatable, there arc other things 
lying beyond his control that can interfere with 
his qfforts to communicate. We will comment on 
both^ides of this problem, although the former 
has been dealt with in Chapter XVII. 

inhibitors to Communication 

"Before going into ways of makings the best use 
of reports, we might look at some of the factors 
that influence their effectiveness as -a -method of' 
communication* All too frequently, readers of 
evaluation or research reports tend to be more 
critical of the research methods, and the ways they 
are reported, than is justified. This attitude arises^ 
in part from a feeling that the obligation to com- 
muDicaUi is solely that of the writer., 

Obvjatte4y, this is an unreasonable attitude. No 
writer can anticipate all the possible situations un- 
dor which the reader mig^ht be able to, or want to 
apply the results of a paj-ticular study.' The reader ^ 
should be willing to make some of the modific^il ons 
or interpretations necessary to relate the study find- 
ings to his own situation. Unfortunately, this feel- 
ing is fairly widespread. It interf-eres with much 
potential use of findings from evalinition and ex- 
tension research. 



Communication in evaluation, more so than in 
^ some other situations, must be a two-way proposi- 
tion. Both the sender and the receiver of the mes- 
sage being transmitted must be active participators 
in the communication process. The writer must 
be able to- send clearly and the reader must be 
willing to receive the message an9 then adapt it to 
■ his own use.- . 

Aside from certain inherent limitations qf the 
written word, there are definite lirnitations on what 
the writer can do to make it easy for the reader to 
"get the findings quickly and effortlessly. The pres- 
sure to simplify involves some danger of misleading, 
the reader. It is. a rare research person or evalua- 
tor who is personally familiar enough with all pos- 
sible applications to apply the findings to all 
possible situations. You must have knov dge of 
the practical situation of the moment lo make 
specific applications. 

Likewise, a pei^n who does not fully understand 
the ^tudy cannot safely make applications to prac- 
tical operating situations which differ in any way 
fi'om the test situation. If the evaluator and the 
program person are. not the same individual, then 
the best substitute is for the evaluator and the 
program person to Avork very closely as. a team 
wliile making interpretations and working out 
practical appIical;ioris of findings. 

Options and Otjllgations 

When it comes to writing his report, the evaluator 
has three options. First, he can descrfbe the situa- 
tion in which the investigation was^done jn suffi- 
cient detail so that the reader can draw his own 
inferences about usefulness of findings for another 
situation. A second choice' is to' make qualified 
generalizations -that still leave ultimate use deci- 
sions to the judgment and imagination of the reader. 

The third choice; is to descr1f>e specific situations, 
either "actual or hypothetical, in which the findings . 
would apply/ Then the reader can decide on the 
degree to which he considers his own situation to 
be comparable to the sample situations, and can 
make his own decisions on how widely to a^ly the ' 
"'Yation findings. 




This gives the writer quite a range of methods 
of presentation. However,- it gives the reador an 
equally wide range of expectations. When presen- 
tation and expectation patterns are not in agree- 
ment» the reader may be displeased with- the report. 
This could be an argument for several presentations 
a.s'we siiggested in Chapter 17, but it could also be 
the occ'asion for a little more tolerance on the part 
of the reader. 

At the same time, limitations of this variety do 
not excuse the writer from doing his best to write 
for the particular readers he has in mind while 
writing the report. This includes the requirements 
of writing as clearly and as simply as possible, using 
tei nis and examples that ^rc within the experiences 
of the readers, using^ppropriate illustrative and 
explanatory materials, and so on. Finally,^ the 
evaluator should try to give the reader the benefit 
of his ow^n unique interpretations, which he alone 
can rnake bej:*ause of his thorough involvement in 
the study. 

Personalized Presentations 

'Che written report alone must be regarded as a 
tess than satisfactory method.of disseminating find- 
ings of evaluations or restfaich studies. Ideally, 
as we suggested in the previoi/s chapter, the pos- 
sible users of the results of an evaluation should 
have the added benefit^ of face-to-face" discussions 
with the evalua'tor. F(i\l benefit from any scientific 
inquiry cannot be achieved without the stimulating 
inTlucnce of personalized communication processes. 
Whenever possible, therefore, you should plan to 
combine both written and personalized presenta- 
tions wheii you desire to achieve a maximum flow 
of communication and understanding. 



Brevity 

Another ^unrealistic expectation held by some is 
that brief sumnuiries of findings are adequate sub- 
stitutes foriul'l reports. The value of this technique 
for reporting diminishes as the complexity of the 
behavior being studied and the number of findings 
to be reported increase. Some things are so involv- 
ed that brief summaries or 6road generalizations 
are likely to 'be more misleading than informative. 

These paragraphs ^.are intended to demonstrate 
that written reports have definite limitations and 
to suggest the nature of their limitations The pur- 
pose of Ihe remainder of this chapter is to offer 
some suggestions -to help you make good use of 
written reports in-spite of any limitations thpy may 
hav*j. The suggestions that follow can aid you in 
making systematically effective use of reports on 
evivl nation research. 



Three Oprerations 

Using reports involves three main operations, 
each* of which has several important steps. The 
three operation.s are, fl) evaluating the quality of 
the investigation that is r'bported, (2) extracting the 
significant findings, and Ci) app^ying the findings 
to' your own' situation. 

Your attention may be drawn to a report in any 
of several ways. The Veport may conic to youi;,desk 
through the mai|;^*ou may come across* it while 
seeking information on a rclated^lqpic; someone 
may call your attention to it to ask your opinion 
about something mentioned in it. 

The first -questions for you to answer to your, 
own satisfaction is, "Would this study or, research' 
project be likely to produce results that could have • 
bearing on my W'Ork?" And'ther question you must 
answer is, **Were the design and the methods em- 
ployed in the study sound?" The- 'answer to this 
question helps answer the next one: *'How much 
contidence can I have in any findings that are 
repWed?'' 

These questions are aimed at something quite 
^different from the report itself or the interpreta- ' 
' tions made by the writer. Their purpose is to find 
out if the study is conducted in a way- that is scien- 
tifically sound so that only facts are brought out 
an.ql so little or no personal bias influences the out- 
come. No amount of statistical precision or careful^ 
interpretation can correct inaccuracies introduced 
through poor design or faulty methods of obtaining . 
'^information. 



Appraising the Quality of a Report ' 

To do an adequate job of appraising the quality 
of a report you need satisfactory answers to ques- 
tions in five main areas. Tne kind of information 
you need can be summarized under these heaomgs.- 

1. Purpose — **What did the investigator try to 
do?'' The important thing here is whether or not 
the' questions he asked or the hypotheses he tested 
are ones in which you are interested. If you are 
not interested in thern, or see no immediate likeliy 
hood of being interest(?d in them, there is little 
point in your going further into the report. 

2. Method — **What kind of a study was done?*' 
and "How was it done?'* are the questions under 
this heading. ^The decision here is whether or not- 
tho gieneral design and the techniques employed 
were such that valid and reliable information of an 
objective nature can be anticipated. The term valid 
•is used to mean really measuring what was sup- 
posed- to be measured; reliable* is taken to mean 
that the same kind of information would be obtain- 
ed if the methods were repeated on other. samples 
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of fho popiilalion. ^ As cxj)lainod earlier, objecftive 
I'livans that information ()r*finclings are not inl'lu- 
t.^nc"C(l,by the personal point of view of anyone. 

;^ Populatiorf— "Exactly who were the people . 
Jrorn whom or about.whoin. information \\'as oi5tain- 
ed?"-^ Vou will need to know this to determine how 
. c ompil able the te.st .situation is to the.6ne to\vhich 
you desire to apply tliu finding.s. While deciding 
the degree to'^'liieh tlie people involved in this 
particular study ^rfio like any group of poo})le you 
are concerned \^Rh. you will be trying toTfincl in 
the description at least sonie ofYhe same character- 
istics that are * miu! in your own groU].\ 

In most cases you will .find .soiiie diffjUJ^ences or 
that cerfain charactei'istics that you t;onsider imi^or-. 
tant are not mentioned in the ix'port. In. that event, 
yf^i will havo to. make pers(;nal judgments oi; esti- 
mates regarding the cliar'acteristics iiot mentioned 
to assign their probable bearing on the findings of 
the study before you can make use df the findings. 

'4. Sample^**What was ' the relationship -of the 
.sample to the whole. |:opulati8n?*' ajid ''Mow was 

• the saniple obtained?" are the related questions 
asked at this point. The sample must remain a ' 
matter of concern, enjn aftei* you arc convinced 
that the population is cjuite siinilar* to your own. A' 
])oorsahip!c can so distort the reported characteris- ' 

" tics of the , population .that the study might as well' 
be i)asetl on "urentirely different population. All.- 
too frequently, infornistion about sampling metho- ' 
d(T!ogy-is not included in the pubiished report^.with,. 
the result that hiany unjustified assumptions of 
comparability arc made. 

5. Timeh'ness — "How ^rec^enfly was the jnforma-* 
tion obtained?" and 'iDoes it apply tociay?|' Favor- . 
able answers to these questions cannot made 
without considering many relat(^d f tutors. A recent 
date of publication or recent, inclusive dates of 



field woj'k da not guarantee data which will serve 
your purpose, Rapi(U'hanges ii? conditions of rural 
If^'hig. and in technological. knowledge' related to 'all- 
aspects af contemporary life, force.us to weigh the 
appi'opriateness c'f the findings in terms of detailed 
aspects of the individual situation rather than in 
br'oad terms of tcmporaTor geographic similarity.' 

Date of publication is.onlj^ a partial check'on 
timeliness.. The study may have been 'completed' 
^luite som^e'time in -the past and delayed montlTs"or 
\eajs because of prior publishing' commitments, 
■budgetary problems of the sponsoring agency, or. 
editorial delays of pne kind or another. Not 
. infj-equently^ studies are based on research com- 
jjleted many years previously or on data sources, 
such a.s tlie census, which fiMiy have been assembled 
ten or more years before the tFme'they are used 
in the. study. 

Summary 

' ''Look^Beforq You Leap," might well be an alter- 
nateTtitle for, this/chaptor. ^rhc niajor emphasis of 
the last lialf is- on wnys -to avoid hasty decisions . 

. regarding comparability and assumed apjDlicability 
to^ydur own situation. We have suggested ques- 
tious in five arcas to which .yoii shoujd be. able to 

'answer.; favorably before you should be willing to 
accept thd^findings " of a. study as appj'opriate to 
your needs. . - ' ' : 

Our?objective in the earlier portions of chi^ chap- 
ter -is to make the point that the primary' respon- 
sibility for communicating may be assigned td the 
writer, but that the reader has an obligation to try . 
to understand what the ,writer .is attempting to 

.connnunicate. Even when a study, or the reported 

■jnformatijoh bout it, may -have limitations,, there 
m*av,be many ways that the. findings can serve as 
a contribution to knowledge if proper interpretar 
tions and allowances are made by the reader. 



* Chapter XX 
IN CONCLUSION 



Ward b\ Purter 



Evaluation is a process of systematic appraisal 
by which \ve determine the worth, value, or mean-* 
ing of something. This something in Extension 
may be a program or part of one, a method or ap- 
proach used in carrying on extension work, or a 
situation such as a community;, a county, or even a 
larger area. Evaluation is a process that facilitates 
effective decision-making. . 

Why We Need Evaluation 

y 

The ultimate goal-' of the evaluation process in 
Extension is the increased effectiveness of our pro- 
grams. Evaluation helps in this respect by telling 
us how much progress our clienteles, have made 
toward their educational objectives. The process 
provides us not only with the results of our efforts, 
expressed in .terms of changes in behavior, but also 
the benchmarks by which this progress can be 
gauged. In addition, evaluation (Contributes to our 
^educational efforts by giving us necpjsary informa- 
tion about situations for planning sound prograrns. 

In the increasingly complex world in. which 
Extension functions, the evaluative process assumes 
.growing importance and significance. Extension's 
•»res{3onsibilities now extend far beyond the -farrja 
horizons of yesterday. Our clienteles are now 
found in cities and towns, as well as in the open 
country. Our audience is better educafecj, bettor 
informed, and more sophisticated. 

Because of the technological revolution in com- 
munications and transportation, people can be more ^ 
easily contacted through various channels, by mass 
media, for example. At the same time people can 
be influencecl by agencies and organizations other 
.than Extension through these same channels. 

Because of these, as well as other changes in our 
society, there is grave need for more systematic 
evaluation in Extension. No longer is it enough 
to know that farmers apply fertilizer. Now we 
should know whether they are using the right kinds 
and amounts, their attitudes toward this practice, 
and various other more or less qualitativ'e items of 
informatioq. Likewise, . sin^^e there are many 
sources of information currently, available to farm- 
ers, we must be more zealous than ever in deter- " 



mining Extension's cp-irtribution to whatever change 
has taken place in our clientele! 

The need to emphasize evaluation in Extension 
is evident. The techniques by .which this process is 
crirried out are less widely understood. Like any* 
other skill, evaluation involves a fund of knovVledge 
and an appreciation o£ certain basic procepures 
an'd "Jvnow-hQw.'^ We hope this manual has made 
it clear that any extension \vorker can develop this 
knowledge and, with some experience, the neces- 
sary skiH to carry on effective evaluation. This 
has been our premise in preparing this manual. 

' - ' ■ " 1) - . 

Basic Elements or Program Evaluation 

Some of 'the basic techniques and, procedures by 
which surveys and formal evaluations are conducted 
have been discussed in the preceding?* chapters. 
Expressed jn much more summary fashion, we can 
visualize program evaluation as consisting of four 
procedural ^^lements: (1) A statement oTf spiecific 
educational objectives; (2) an inventory erf whatever . 
was done in the name of the program Ho achieve 
the objectives; (3^ an appraisal of the, situation 
beforp the start o£»the program — tliat is, benchmark 
determination; and (4) subsequent measurement 
of the situation to determine progress:,and accortr^ 
phshment.* These can be thought of as morie or less 
consecutive phases or steps in evaluating any given 
program. 

Pin-Pointing Objectives— The fiVst step irt^any 

program' evaluation calls for an appraisal of specific ' 

progvam'^' objectives. In this instance,' we' must 

answer the question, ''What were we' trying to* 

accomplish?'* The "What" must be specific enough' 

to be measurable; "and it must spedify the changes 

in the situation (i.e., in the people) that the prograrn 

hoped to accomplish. It must also indicate the' 

particular .audience involved. * ' \ ' 

- .. 5 . \ 

Program Activities and Methq'ds — In this,, the 

second phase, >ve must account for all actions taken^ 
in the namejbf the program. This is not always, a - 
simple task, particularly where many agenciejSMiP.d ^ 
organizations are involved. Because of ^ur failure 
to do this adequately, we in Extension are some- 
times in the uncomfortable .positioiihof being credit- 
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0(1 with, or appearing , to claim credit Toi', accorn- 
'plishiiicnts [\>v which \vo should" not iCgltinuitcly 
cxpt^'ct ctc'dit • ^ ' . , 

This (iirriciilty can frequently bci overcome. \u'0- 
vidini; there i.s citjse cooporatipn between bothpro- 
grani persoruieP aud the evaluator.s. .. Tl)i.s • is' one 
instance wherr thci i^^ht hand must know what the 
left liand'is^doiii^. , ' " " * ' 

Estabh'shing Benchmarks — One Of the most 
essential stejis. in any' ])rogram evahiation is an 
accurate appraisal of the .situation before 'llie pro- 
gram rtarts. Unfoi'tu'nat^ly, this appraisal is fre- 
. quently ifeglected until sometime after the prograin 
is uncier way. We recomniend appropriate bench- 
mark surveys at the beginning of all programs. " In 
any event, there hds to be some (Evidence of what 
the i}re-progL-am situation. \^ as like before we can 
decide how ftiuch pri^gress has been inadd. 

..Determining Program Progress and A^ccomplish; 
ments-^The fourth ! sic step in ev^?.luation is to 
determine progress or end results. Findings or 



conclusions of this nature must obviously bo inters 

prctcd on- the basis of,i>i-ogram objectives— i.e., the! 

amicipated educational changes. They niust also 

^ be expressed in terms of the pre-de^ermined bench- 

rpark, . Further, since we'are evaluating extension. 

work, we* mus{ -••onsidcr the activities and methods 

* used i-n pui^surng prSgram objectivesT Only in this 

;way can we correctly and safely appr^iise the- siic- 

.cess of pur efforts. 
if 

The Challenge ^ 

. ; Evaluation js a tool; it is hot ah end in itself: It^ 
js/the mean5 by which extension programs and 

• jnethdds can be made more effective. - If an eValua-' 
tion is to be effective, the e;itensi'on workers in\felv-^ 

, ed nfust incorporate the f inklings into their on-going , 
prograni. ' • , ' ^ • " 

All extension workers evaluate.- When they 
so systematically and objectiv^eJy, they de.iVj^ 
increased personal satisfaction from the sui'e knovvi- \ 
edge that their work h:. achieved some measure 
of success: 
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Appendix A 



ILLUSTRATIVE EVALUATION OUTLINE 
EVALUATING A PASTURE 
IMPROVEMENT PROGRAM 

Frvd P. F)'fffrh( !/ 



The following is an illustrative GUlUnr for evaliuUin;^ the 
results of teaching in carrvin?; out part of an extensio: 
pro'iiam. The main purpose of this evaluation is to find 
out wliat decree the teaching' on improving pastures has 
been effective. To what extent were the objectives of the 
teaching efforts reached? 

The decision to be made on the basis of the evaluation 
was, • Shouid the emphasis on pasture improvement bo 
continued or are pasture improvement practices well 
enough established thai time and effort can be shifted to 
other pressing problems?" This outline consists of leu 
steps: 

1. Backj^round Situation — The need in the county out of 
which the pasture improvement program ^revv. 

2. Trouram planning— The procedure by which the prob- 
lOMT KOt into the pro^^rani and plan of work. 

3. Objectives of the Pro^*ram — The results expected from 
. the pro^^rani. 

4. Extension Tt»aching Methods — This shows what was 
taught and l.ow. 

5. Questions to Be Answered by the Evaluation — These 
are the specific questions to be answered. 

6. Sampling the Population — This inch!(U»s the people to 
whom the pasture improvement program applies and 
the sample. 

a. Defininj4 the population. 
I). Solectin^^ the sample. 

7. How to Gel the Ini'ormrtion — This step is concerned 
with tho procedure for obtaining the information 
needed, 

8. I*reparation of Questionnaire — This is the form on 
which the replies ol those who answer the questions 
are recordetl. 

9. Tabulation, Analysis, and Intcrv""<^'tation of the Infor- 
mation — In^this step the informally.'- on the question- 
naires is ad^Iyzed. organized and summarized to deter- 
mine whatMt r; '"^ns for the proj^ram. 

10. Ufcc of the P'indin;-> -The application ot the findin^^s 
fib program adjustmenl. 

Steps Illustrated 

v! Background Situation-.-The need in the county out of 
which the pasture improvement program firew. In Green 
County, many farmers have been changing from row crops 
to livestock but their pastures are inadequate. Amon?^ 
other things the pastures need Ume. fertilizer and rcseed- 
ing. The county agent had worked on this with some 
individual farmers but no concerted effort had been made 
over the county. 

2, Program Planning — ^The procedure by which the prob- 
lem got into the program and plan of work. This situation 
was presented at the annual meeting of the county exten- 



sion program planning committee pointing, out that tht 
a!)ove practices were among the important ones recon; 
mended !)y the experiment station to build better pastui:ei: 
After some discussion' the committee selected these prac 
tices for* emphasis in the program. The county extensioi 
staff was advised to spend considerable time on this phasi 
of the program. 

3. Objectives of the Program — The results expectci 
from the program. From their knowledge of the situatioi 
and on the basis of the recommendations of the committee 
the county extension staff formulated the following objec 
tives of the program. These were the changes they hopcc 
to bring about in the farm families in helping them to hclj 
themselves. 

The objectivek should be clearly and definitely stated 
V^ague objectives must be analyzed until they are observ 
able evidences and made tangible. They may he wha 
people know, do, or their attitudes. 

Tile county extension staff decidod upon the following 
objectives of their teaching program. 

During the next three years to get at least 400 of the 50C 
livestock farmers to improve their pastures, thereby inak 
ing it possilDle to support more livestock. P^our specific 
objectives wer^' set down. They were the three practice: 
stated earlier .ind an additional practice — testing the pas 
ture soil; lo get the I'^ostock farmers: 

a. to lime their pasture,*' 

b. to fertilizer their pastures 

c. to reseed their pastures with recommended seedings 
(L to test pasture soil 

These, then, arc not only the objectives of the teaching 
program but are also the results to look for in the cvalua 
tion. 

4. \Ex»ension Teaching Methods — This shows what wa = 
taught and how. ^ These two columns show what was done 
to reach the objectives of the program.- In the second 
column arL' the Icarning'experienccs provided to influence 
farm families toThange. ^ ^ 

WHAT WAS TAUGHT 

a. - Liming and fertilizing would produce a rnore luxur- 

ious growth of grass. 

b. Resceding will produce a more complete stand of 
grass and utilize the acreage more Tully. 

c. These practices are recommended as the result of 
research at the experiment station, 

d. Some farmers in the county have tried them out with 
success, richer pastures supporting more livestock. . 

e. How they can get and apply lime, and fertilizer, the 
use of soil tests, and how to get their soil tested. 
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HOW IT WAS TAUGHT 

a. Circular iotti*r sent all livestock farim-rs m the 
i-oiinly caliin;4 th<'ir atlciUioii to this situation and 
invitini^ Ihcni to coint* to a incctiiij^ in tlu-ir part of 
llu* fr)iinty. 

h. Mcctuiiis in various parts of the coiinl\'. usin^ visual 
anl*;. and testimonies oi' fariniTs who had aduptc:! th.c 
practices. 

c. I.)ernoiistrations on farms in viirious parts of tlie 

d. Pamphlet on iniprovinL; oast tires. 

e. Timely news articles and circular jetlers. 

f. Three major radio talks an<l short references from 
time to time on the radio. 

!4. E'xhihit at the county fair. 

h. Posters at lime, fertilizer and swd dealers. 

i. The home deinorislralHjn aUi-nt mentioned and talked 
ahoiit these practices at hoine demonsl ration cliilj 
meelin^'s. In 4-} I Club work the prncl ices were dis- 
cu.ssed, particiihu'ly in the livestock clubs. 

5. Questions fo Be Answered by ?he Evaluation — These 
are the specific (pieslions to be answered. The (piestions 
to be answered were, of course, the de.uree lo which the 
olijeclives were reached (the results) plus any other per- 
linent results that may he expecteri. These fjuestions- were 
formtiluted a^ the ones on which information would be 
collected, 

a. Hc)\\ many livestock fanni-rs limed their pastures'^ 

b. How many livestock farmers fertilized their pastures? 

c. Iliivv many livtvstock farnu^rs reseeded their pastur(\s'' 

d. llo\\' many livestock farmers had their soils tested? 

e. Il(*w many f;irniers increased their niimber of live- 
stock? 

f. What kind of livi'Slock were iricreased and how man>' 
of eadi kind? 

1^. What was th'* estimatcMl increase in '.alue of live- 
stock 

h. How did this compare with the cost of the a<loption 
of the practices? 

Thi'se and perhaps other fpiestions would In* answer^! by 
the evaluation. A <piestioniiaire would l)e prepared to .uet 
ans\ ers to the first six questions.- The analy.sis of the 
replies woulit consider the farms on whicli the practices 
were aciopled for the first time durinj4 the three year 
perio<l and the livestock increase per acre. 

6. Sampling tbe Population — This includes the people to 
whom the pasture improvement proL^ram applies arul the 
sample for the evaluation. 

Defining the Popi/lation — The jiopulation was. of course, 
the 500 livestock farm families in the county. These are 
the people beinu taiij^ht. They arc the ones at whom the 
^J?.L2iiram was aimed and who are in a position to do some- 
thm^Lj about the practices. 

Selecting the Sample — You want to j^et the answers to 
your questions as economically as po.ssiblc least time ;in(i 
effort I an<l with as much accuracy as you ne( d. A sample 
of 100. chosen at random, from the 500 livestock farmers 
will uive fairly accurate ri'sults and at the same tinu' 
reduce the amount of lime anci effort spent on the evalua 
tion. 

There are several ways of seloctinq a sample. In ti;:s 
case every fiftli name on thi' office's alphabetical list of 
livestock farmers in the countv was sLdected for the sami)le. 
Selecting' at random docs not mi'an :>electinf: hnphazardly. 
It means a systematic method without subjective bias. 



7. How to Get the Information — This step is concerned 
with lh(^ procedure for obtainin.L: the information necdcci 
in answering the (piestions. ^ 

J Ik' inf()r)nali')n about the practices cmild be obtained 
l)y s.'ndin^ a mailed <iU(»sLionnaire ami covering letter to 
thi' samplr of Kjo li\'e.stoek families. This look.s like the 
"asie>t uml quickest way bui often is not the ea.se.. Many 
families don't return the <]uestionnair(\s without one or 
iitori' foliuw iipTetter.s which extends the time for ;.{cttin;? 
the infoMnation and increases the follow-up work. Yoif 
inu>t have all or practically all of the JOO questionnaires 
returned; otherwise your sample n]:\v bee. .ue biased and 
the error increased. 

.Another method is to interview each of the 100 livestock 
ramdies in the sample. This method .sounds at first ulance 
to \)(- too cumbersome and tin;econsumin^. On further 
consuieratiun U can be nuide (pule practical. The three 
a.L{ents in the county could diN'ide up the 100 families; each 
ai^ent having' 33 or 34 families to interview, taking about 
3 days or so. The interview not only serves to collect the 
information in a very reliable way; it also serves as a 
heli)ful farm and home visit and as a follow-up of a pro- 
i4r;im, thereby showin;,' the interest of the a.Lients which 
would be appreciated by the families. 

It could also he done ])y members of the county projjram 
planninj^ committee after they have been properly tr.Tincd 
to do so. This has certain decided advantages for the tvpe 
ul evaluation u.scd in this e.xnmpkv 

8. Preparation of Questionnaire — This is the form On 
which the replies of those who answer the questions are 
recorded. It may be a questionnaire to mail or for use by 
interviewers who visit the farm for the information. In 
any case it is a form on which the information is recorded. 
The example , which follows was partially designed as a 
mail (piestionnaire. 

Questionnaire (Example) 

PASTURE PRACTICES IN GREEN COUNTY 

(Date) 

Dear Sir: « 

This is a ma)icd questionnaire sent to lOU livestock 
farmers sclcf ted as a cross section sample in Grten County 
to survey the pasture practices used. 

Your reply will be greatly appreciated since the tqIi^ 
ability of the survey will depend upon having every ques^ 
tionnaire returned. 

We believe the questions can be easily answered in about 
10 minutes. 

We will appreciate your cooperation very much. 

Sincerely yours, 

(signed by the county agent) 

1. Do you lime your pastures? {Check) \ 

a Yes; b No, 

2. If Yes. w hen did you start to do sc? 

a. _ During the past 3 years, 

b lief 010 3 years ago. 

3. Did yen do more liming of your pastures during; the 
past 3 yoaKs than you did previously? 

a Yes; h No. 

4. Do you fcriilize your pastures? 

a Yes; b No. 

5. If Yes. when did you start to do so? 

a During the past 3 years. 

b. ... . _Beforc 3 years ago. 
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iS. Did you fertilize your pastures more f^i/ir,'/ the prist 
3 years than you did previously? 

a Yes' b... Xo. 

7. Do you reseed your pastures? a Yes; b No. 

8. If Yes, when did you start to do so? 

a During the past 3 years. 

b Before 3 years a^o. 

9. Have you resecdcd yo,ur pastures more during the p:isi 
3 years than you did previously? 

a Yes; b.. „i\o. 

10. Have you had your pasture soils tested durinfjj the past 
3 years? a Yes; b No. 

11. How many of each kind of livestock do you have on 
} our farm? 



Now 

(1) 



a. Dairv cattle 



Three 
years ago 
(2) 



b. Beef cattle 



C. Sheep 



d. Ho^,^s 

12. How many acres do you have in pasture? 

a __Now; 1?.- — Three years ajj;o. 

13. Have you heard about the pasture improvement pro- 
jLjram in ihe county? a Yes; b No, 

9. Tabulation, Analysis and Interpretation of the Infor- 
mation — In this stop the inroi-mation on the questionnaires 
is analyzed and summarized to determine what it menns 
for the pro^iram. 

When the questionnaires are completed, one or more of 
the agents can edit them, prepare a "typewriter tabulation" 
sheet and show a secretary how to tabulate the replies. 
These will be analyzed anfl summarK^d by the agents and a 
'?port prepared. The following is an example of "type- 
writer tabulation" sheet. 



10. Use of the Findings — The application of the findin 
to program adjustment. The report will be presented 
the county program planning committee- for it-s study a 
discussion and to determine the extent to which "imprc 
ing pastures" should continue to receive emphasis in nc 
year's county extension, program. The results of the ev 
uation will give the agents some clues as. to the cffccti' 
ne.ss of the leaching methods used. 

The information will serve as a topic for a circul 
letter, radio program or news letter whicli has certa 
pubJic relations vaJues. H'hc agents will have the sal 
faction of knowing very definitely the degree to whi 
the objectives of that phase of the program were acco 
plished and of their contributions in the county. T 
following arc examples »P^517^T5^ft^uLcharts. 



TABLE FOR REPORT (Example) 

Percent of Livestock Farmers Using 
Improved Pasture Practices 

Pasture Practices 

1. Liming 

a. Lime pasture now ^ 80 

b. Started liming during past 3 years . . 60 

c. Did more liming during past 3 years . . . 70 

2. Fertilizing 

a. Fertilize pastures now 75 

b. Started fertilizing during past 3 years .... 70 

c. Did more fertilizing during past 3 years . . 75 



Total Questionnaires Received — 100 



TABULATION SHEET (Example) 



County- 



Question' 
naire number 
and /or name 



1. Lime 
pasture 



a. Yes 



NR 



2. Started 
liminu 



a. Past 
3 yrs. 



b. Before 



NR 



3. More liminQ 
durino past 
3 years 



a. Yes 



b. No 



NR 



4. Fertilire 
pasture 



a. Yes 



b. No 



NR 



Thts iltu&tratet th« tabulatino procetture. Count all th^ X't in column la and write that number at the bottom of column la. 



Total 



to 



ts 



70 



20 



10 



70 



20 



75 



2% 



5. Started fertiliiino 



a. Within 
pist 3 
yean 



to 



b. More 
than 
3 yrs 
aoo 



15 



Don't 
know 



X 
X 



15 



I. NR means "no respome" 
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CHARTS FOR MbETING OF COUNTY EXTENSION 
PROGRAM PLANNING COM^)ITTEE (Example) 



Pasture ■ 
Practices 



Percent of Livestock Farmers Using 
Pasture Iin pro vem ent Practices 
More Since Program Committee's 
Decision Three Years Ago. 



Percent 



Liming 



Fertilizing 



70% 



15' A 



20 

L_ 



40 

— 1-^ 



60 

L_ 



80 



100 

— I 



Pasture 
Practices 



Liming 



Fertilizing 



Percent of Livestock Farmers Using 
Pasture Improv.ement Practices 
Before and After Program Commit- 
tee's Decision Three Years Ago. 



Percent 



20 

U- 



40 

L_ 



60 



80 



100 

— \ 



Total 

Before 20% After 60^1 

80%/ 



Before 5% After 70^^ 

75% \^/yy//yy?7////y//yy//yyA 



[ 101 ] 



EKLC 



113 



Appendix B 



EXAMPLE OF AN INFORMAL 
EVALUATION 

Mrs* Laurd K. SabroHkii 



This example of an informal evaluation illustrates the 
importance of plannin;^' ahead of time and involving learn- 
ers to assure fullest interest and m^iximum benefits. • 

Evaluating a Series of In-Service Training Meetings 

The audience at these meetings was about 30 or 40 exten- 
sion workers; the speakers and discussion leaders were 
authorities in the various subject-matter areas; the plan- 
ners were extension workers wit(i training responsibilities, 
This is the procedure that was followed: 

L Each speaker and discussion leader was asked to pro- 
vide the planners, before the mertinss started, \yith a 
statement of the educatiorial objectives he had for his pixri 
of the projiran;. in terms of what he expected the learners 
(exttmsion workers i to ^'el out of it. The. speakers and 
discussion leaders were completely cooperative in provid- 
iiil; this information. 

2. Near the end of the series of mcetin^^s. the audience 
was formed into working groups, and each group discussed 
ami decided on the most important things it-^iad learned 
from each lalk or discussion. 

3. The evaluation discussion leader placed, on. a black- 
board the things learned by the audience from each talk 



or discussion. He placed these on one half of the black- 
board. Then he wrote on the other half of the board the 
expectations of the speaker, or discussion leader. 

4, The total audience then reviewed the similarities ind 
differences between what the speaker or discussion leader 
expected and what they thought they had learned. All 
speakers and discussion* leaders who were not involved in 
other professional duties at the time were present for this 
evaluation session. 

5. Members of th^c audience discussed what they were 
expected to learn, why they ftUd learned what they were 
expected to learn from some, speakers and discussion lead- 
ers, and why they did not learn the expected things from 
others. The evaluation leader brought out that they were 

.following the approved method of evaluating an educa- 
tional effort; (1) Doing so in terms of the objectives of the 
• teacher/' (2) involving the "learners," (3) analyzing the 
results to sec reasons for success and failure, and (4) look- 
ing ahead to, occasions for applying the results to similar 
situations, • 

This evaluation procedure not only illustrates a means 
of evaluating properly, but also provid%£ a method of 
rcemphasizing the points the audience has learned, and of 
bringing to them points ^that they have missed. 
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Appendix C 



PROFESSIONAL HELP IPOR THE ^ 
EVALUATOR 

Josepli L. Matthews i 



The potential area from which to select an evaluation 
project in extension teaching is as broad as Extension 
itself. Any project you mi^hl select eoulcl deal with any 
one phase, or complex of phases, which could range 
from one extreme of extension activity to another. At the 
same time, no extension worker can hope to be an author- 
ity in all areas of the work. Therefore, any of us who wish 
to pause and assemble information that will help us im- 
prove our work need have no feeling oD embarrassment 
over not having all necessary knowledge and skills to 
follow through with appropriate evaluation. 

For example* many of us are trained in special areas of 
agriculture or home economics. Once we are on the job, 
however, we find that many of our administrative and 
coordinating tasks require considerable understanding of 
the social sciences. We need to know techniques for 
identifying differences between persons who should or 
could be reached and those who actually arc reached by 
our teaching methods. We need to know why certain 
groups among our clientele do not respond so well to our 
programs as we believe they vshould. We need to know 
the differences between urban and suburban families in 
our county, and how these differences may influence 
their demands on oOr services. 

We need to know how best to approach certain families 
or individuals — how to work through their needs, values, 
motivations, ambitions, knowledge and understanding. Wc 
need to know which methods will work most effectively 
with which groups — how to approach 'them how to make 
them interested, how to hold their attention, how to en 
courage them to follow Ihrough la completion, and how 
to carry on by themselves. ' 

Although these are everyday concerns with most f us, 
it is easy to see that gaining the necessary information in 
the proper form to make the kind oi evaluation that would 
help us in our work would requivq more than routine 
understanding of evaluation methods and techniques. In 
spite of the certain value such knowledge would have for 
us, the size of the project makes us pause to consider the 
question, ''Should I undertake evaluation activities that 
are beyond my present competence?** 

Such a question implies that we feel we hare to *'go it 
on our own" in evaluation. Actually, thjye are many pro- 
fessional resources which arc at our disposal. The kind 
of question we should be asking ourselves is, "Are the 
needed resources and technical competences available to 
help with the evaluation I want to make?" 

Whpn we state the question in this manner, it is much 
easier to gain the confidence to start for, in most instances, 
we will find that all the professional help we will require 
is at oLir disposal. All wc have to do is look around and 
take counsel where needed from persons who are'compe- 
tent in the areas that are difficult for us or unfamiliar to 
us. With t^cir help, we can move on to appraising what 
resources wc will need and finding out whether or not 
they are available. 

The' final decision to undertake an evaluation, and how 

[ 
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extensive the investigation shall be, will depend upon the 
availability of technical help and other special resources 
that must be marshalled for the job. Technical help is 
essentially a personnel problem — a matter of contacting 
the right people and enlisting^their knowhow. The other 
resources consist of time allocations by persons involved 
in the evaluation, and any special materials or facilities 
that may have to je assembled. 

As in all investigative activity, the evaluation project 
finally outlined and executed will be a compromise be- 
tween what we would like to do and what wc were able to 
do. We find out sooner or later that others aro not so 
convinced as we are as to the relative importance of the 
evaluation we contemplate and that they have other ideas 
for using limited available common resources. 

Before we complete the evaluation task we can expect 
to make numerous compromises for reasons of economic 
or practical expediency. However, compromises provide 
practical educational experiences in resourcefulness and 
persevercnce and should not be taken as reason for drop- 
ping a research er e\^luation project unless they invalidate 
the basic objectives of our efforts. 

If you have never been involved in a systematic evalua- 
tion activity, you may be at a loss as to where to turn for 
the help you need. This confusion should not last long, 
however, as wc in Extension arc fortunate in that we have 
ready access to a wide variety of professional guidance 
and help. 

All of us have access to the technical and professional 
resources of our own land-grant institution, and some of 
these resources are exceptionally helpful. Of course,- 
there will be some variation in the particular assortment 
of competences represented in a particular State extension 
staff, resident teaching staff, or experiment station staff, 
but you can count on finding someone who can help you 
get a start, or who can tell you where to look if he does 
not feel qualified to advise you. 

Availability of such persons may be limited by other 
demands placed on their time, but most of them can find 
time to get you started with your evaluation. Another 
factor.which may interfere with their effectiveness insofar 
as your work is concerned is the distance from the institu- 
tion to your study area. If you can run in or your con- 
sui^'*;l * can come out in a few minutes, you can get together 
about as often as y.ou care to. If the trip is one which 
will require hours or dj^ys, both of you will have to plan 
your meetings carefuUS^, and vyorK up agendas for when 
you can get together./ At other times you will have to 
resort to correspondence or to working with substitutes 
he may be* able to r^ommend. 

Many times he can suggest specialists and facilities of 
his institution which are located off-campus but which may 
be more convenient to your study area. There are many 
facilities of this nature which you can feel free to draw 
upon. No doubt you will be amazed when you learn how 
many of them there really arc. For the balance of this 




appendix .section, however, we will confine our remarks 
to facilities available ffom the land-j^rant institutions and 
fhe Federal Extension Service. 

State Extension Staff 

You are no doubt aware of the many speeic^.lists on the 
Cooperative Extension Service staff of your State but you 
may not be aware that over 20 States have a specialist in 
extension research on the staff. Your Stntn may be one 
of them. This staff member is the proper initial contact 
for aid in any kind of evaluation project. Ir. most instances 
he has had special training in the techniques of evaluation 
and is quite likcly.-to be highly competent in social science 
research. 

lie can assist you personally in plar^ning and following 
through on an evaluation activity, in addition to the help 
he can give you personally, he is in touch with other 
individuals in resident research and teachinC who have 
the skills you may find it necessary to draw upon. 

Most of the 30.odd State personnel training ijpecialists 
have some competence in evaluation and research in gen- 
eral. In the absence of a State extension re^varch person, 
the leader of the training program is a f^ood person to 
contact. 

Most State staffs have one or more ryerscns- who bnve 
participated in formal evrj^luation work, or who have takci. 
the evaluation course at a regional exteuiiion summer 
school. If there is no extension research or training per- 
son on the staff, a person with this kind of background 
can be consulted. 

Resident Teaching Staff 

All resident teaching staffs at the land-grant institutions 
have individuals with backgrounds of research in educa- 
tion or the social sciences. Usually there is someone on 
the staff who has had research experience direclly related 
to almost any conceivabie evaluation problem we might 
have. All you have to do is find the proper person and 
make .suitable arrangements to get the help you need 
from him. 

Good persons to seek out when you need help on evalua- 
tion stvc the ones teaching research courses ox those 
engaged in research work in the various subject-matter 
departments. The more experienced staff members in 
departments of mathematics or statistics, psychology, edu- 
cation, economics, sociology and anthropology can make 
helpful suggestions regarding evaluation methodology. 
Although their suggestions would have to be modified to' 
fit your particular needs, they should be helpful in extend- 
ing your understanding of general approaches and basic 
investigative techniques. 

Experiment Station Staff 

Every experiment station has on the staff persons whose 
work is planning and foll9wing through on a wide variety 
of research projects that involve essertially the same prob- 
lems of design and execution that ^ve encounter in our 
evaluation projects.. This is particulirly true of the people 
engaged in sociological research, sampling methodology, 
questionnaire design, and data collecting and processing. 

Slost experiment stations have specialists who devote 
most of their time to working out coding plans for ques- 
tionnaires and schedules for either hand or machine tabu- 
lation. They also serve as consultants on questionnaire and 
schedule designs. Usually there is someo.nc who is experi- 
enced at preparing tabulation plans in sulficient detail so 
that the followup work can be done by- clerks or machine 



operators. All of them are available to advise you in p-^o- 
ce.ssing your own data for analysis and interpretation. 

Most of the experiment stations have modern electrical 
machines for tabulation and other analysis operatfons to^ 
speed up data processing. In many .States the facilities 
are adequate to handle almost any statistical job likely to 
be undertaken at the institution. During slow peripiiiS 
they are made available to outside researchers. At^-^uch 
times you may be. able to arrnnge for use- of the ma<;nincs, 
together with the services of the operators, to^ process 
your data. / 

Adequacy and availability of facilities and qualifications 
of staff memBers and technicians will vary frpm one insti- 
tution to another, but all of them will be able to help a 
great deal. Most of the time the resources are more than 
adequate for routine evaluation work. This is particularly 
true of the kind of evaluation wc would do by ourselves 
or with the cooperation and assistance of a few colleagues. 

Even overloaded resident teaching and experiment sta- 
tion personnel are willing lo advise us to the best of their 
ability when \, c approach 'them tl^rough proper channels. 
When they are convinced we are i^ncero in our efforts at 
evaluation, they will be most tolerant and understanding 
of our comparative incompetence, and will go out of their 
way. and make personal sacrifices to be helpful. . 

Division of Extension Research and Training 

This Division n the P^ederal Extension Service is a 
service unit thr.t has as one major responsibility, the 
national leadership in ' extension research and the staff 
training in evaluation. This combined function is accom- 
plished by coordinating extension research and, evaluation, 
by training and assisting State research persons with their 
work, by providing courses and training sessions in eval- 
uation, and by consulting und participating directly in 
evaluation activities at all levels. 

The si/e of the"' staff and the facilities available to it 
limit active participation in evaluation activities to a re- 
stricted number of major national, regional, State and 
county studies. Most of these are done as cooperative 
projects between the State or regional group and the 
Division. All of them must be of the pioneer variety to 
some degree, although a certain amount of replication 
- occurs. 

The major contribution of staff members is made ■ 
through personal consultation in the States or through 
correspondence with individuals to advise "Ihcm on eval- 
uation activities and extension research projects. Assist- 
ance provided may consist of giving suggestions about 
sources of needed ihelp, supplying materials, advising on 
specific phases of the work, or reviewing plans for all 
stages of the contemplated evaluation sequence. 

Requests for assistance frpm Division personnel by. 
workers in a State should ic made through the St?te Direc- 
tor to the Director of the Division of Extension Research 
and Training or to a particular staff memtfer of the Divi- 
sion. If there is some question as to the identity- of the 
^ person who can give the desired help, the request should 
be made to the Director who will refer it to the proper staff 
member. Otherwise, the State Director can communicate 
directly^^with the proper staff member. 

The general categories of help that can be given by the 
Division are listed as major topics in the accompanying 
outline. Under each major topic are listed the specific 
k^nds of assistance that are. offered in that area. 

1. Defining the problem and determining necessary re- 
sources. 

a. Spelling out the problem and the general nature of 
the information that will be needed. 
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b. Investii^atin^' resource needs and possible Ways of 
meeting them. 

c. Analyzing 'pie time requirements and f)rcparing a 
preliminary ealenclar of work. 

d. Analyzing the nature and extent of tho cooperation 
that will be neec'ed from other extension workers. 

2. Desigring'thc evaluation plan. k 

a. Sjielling. out what was taught that is to be evaluated. 

b. Deiiiung what aspepts of the teaching action are to 
be evaluated. 

c. Deciding upon objectives of the evaluation. 

d. Deciding what evidence is to be collected. 

3. Preparing research forms. 

a. jpeciding what questions to ask and how to phrase . 
them. 

b. Developing the record format. 

c. Preparing instructions for handling tl\e questions. 

4. Planning tabulations and the analysis. 

a. Preparing the coding plan. 

b. Deciding on the tabivlation method. 

c. Preparing tabulation forms. 

d. Deciding on statistical techniques. 

5. Preparing the sampling plan. 

a. Defining the population. 

b. Determining the sampling fraction. 

c. Preparing the sampling instructions. 

6. Conducting the field survey. 

a. Selecting interviewers or recorders. 

b. Trainiijg interviewers or recorders. 



c. Supervisin g field party. 

d. Editing srihedules. 

7. Prep::ring the report. 

a. Planning thej*eport outline, 

b. Analyzing fhe data and' drawing implications from 
them. 

c. Writing the report. 

d. Publishing the report, 

8, Getting- findings disseminated and used. 
I a. Interpreting the "findings, 

15. Drawing iinpiications for action programs. 

c. Planning the disseminating activities. 

d. Setting up r<?porting situations. 

In addition to the direct help that members of the 
Division staff can give, there are 'numerous duplicated 
materials on cvali!Iation in Extension that are availably for 
distribution. These materials consist of examples of ques- 
tions or complete questionnaires, various kinds of evalua- 
tion check sheets, tabular layouts and recording devices, 
and brief statements on methodology for the different 
phases of evaluation. They also have samples of different 
kinds of completed evaluation reports. 

Staff members are prepared to answer most routine 
requests for suggestions about how to handle specific 
evaluatiun problems. In many cases they can provide 
appropriate example materials to^ help make their sugges- 
tions clear. When the problem is a new one to them, they 
will do all in their pow^r to work out a practical solution. 
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Appendix D 

EVAIUATION REFERENCES IN 
EXT£NSiON SERVICE REVIEW 



From time to lime the Extension Service Review contains 
articles on the need for evaluation, planning evafuation. 
methods of evaiiiaUon rosults of evaluation, and uses of 
the results. Some of ihcsu references arc listed below. 

January, 1955 ' • " 

U Gailuii. Gladys and Matthews, J. L.— Appraising 
•Farm andjiome Unit Approach, p. 19' ' * 

April. 1955 ' ^ • 

2. Cowlitz County, Wash. — Who Are the Members of 
Home Demonstration Clubs? p. 84 

May. 1955 • . 

•3. Sahrosky. Laurel K. — Some Ways to Hold Your 
• Young People^ p. 101 
4. Clayton. Leon 0.— Factors for Successful 4-H Club 
Work. p. 105 

September: 1955 

,5. Matthews, J. L.— What Is Evaluauon? p. 171 

6. Fificld. Marguerite and Wtjitman. George E. — After 
We Interviewed 346 F'amilies We Revamped tho 
County Program, p. 173 . ' 

7, Knutson. Audio L. — How Do You Measure Progress^, 
p. 174 ' ) 

•^*8. Procter, Harriet — Ou^ Time — Use Study Was an Eye 
Opener, p. 176 
9. Harris, Margaret E.— We Take Good Look at the 
. Women We Serve, p. 177 
lo: Fessenden. Jewell— Tools fof Testing, p. 178 

11. Raudabaugh. J» Neil — Program, Plan of Work, Pro- 
gram Evaluation, p. 180* 

12. Ueland* Gale—Half of Louisville Listened, p. 183 

13. Clark. Robert C— The Home--Focus of Our Re- 
search, p. 184 

14. Frazier, Leslie — Our Families Wanted Facts, p. 187 

15. Schruben, Luk M.— Compounding State Research, 
p. 189 . . 

February,. 1956 ^ 

16. Pierson. Arlie A. — One Way to Measure the Value of 
Mass Media, p. 45 

March, 1956 

17. Bianchard, Evelyn 'and Shipley, Fern— Four State 
Stafr Pool Ideas for a Revised 4«H Food Program, 
p! 55 

May. 1956 

18. Copp. James H.—Why Some Stay In i>ind Others 
Drop Out of 4-H Clubs, p. 97 

19. Kern. K. Robert— Iowa Folks Fou»^d th': Answer to 
F?cts~So What?\^P^ 100 

20. Marchant. John D. and Rowe, M. Pai.Lnc— Our Time- 
Use Study, p, 103 • 

.June, 1956 

21. FES Task Force— Earmarks of Farm and Home 
Devefopment. p. 109 



July, 1956 . ^ 

22. Dewar. Carmen L. — Iowa Women Learn to Serve 
NutriticHis Meals, p. 132 

23. Crosson, Mary Alice— Indiana Specialists Test Their 
Teachin'g Skills, p. 134 - ' 

August, 1956 

24. Sevoian, Lucile Hieser — Complete — Not Complete; 
A Satisfying .Project Experience in 4-H. p. 149 

September, 1956 r 
' ' 25. Ross, John E.— More Than Brick and Mortar, p. 157 

26. Lonj^sdorf, Lisle L.—Facts and Dreams, p. 163 

27. Several Counties— Let's Tackle One Problem at a 
Time, p. 166 

28. Wyoming Extension Specialists — ^Analysis of Needs 
— A First Step in Program Projection, p. 170 

29. Swanson, Harold B. — Early Results of Program Pro- 
jection, p. 1^2 

November, 1956 

30. Pope, Edward V.— Do You Know Yourself? p. 204 

31. Beers, Howard W. and Rheinwald, Hans — Evaluation 
Takes Root in Eurqpe. p. 206 . 

32. flchenor, Philip— A Hazard Hunt. p. 207 

December, 1956 , ^ 

33. Dodge, ^W, A. and Bradfield, Alex — Vcrmonters InT- 
prove Milk Flavor to Increase Sales, p. 221 

34. Stanislaus County, California — What Extension Help 
Do Women Want? p. 235 

January, 1957 * • ^ ' 

35. Erickson, Anna Jim— Listen to the People Speak. 

p. 11 . ^ 

March, 1957 

36: Bergen County, New Jersey— Food— No. 1 in 180,000 
Budgets, p. 63 

April, 1957 

37. Reeder, Robert— We Took Another Look. 77 

38. Liveright, A. A.— What Makes Them Tick— My 
Observations of Two Successful Community Organi- 
zations, p. 79 

May, 1957 

39. Rutledge, Uoyd L.— Clues to 4-11 Club Program 
Vitality, p. 105 

40. Dildine,* Glenn C— Your Citizenship Improvement 
Study, p. 107 . 

Sabrosky, Laurel K.— Let's Be Challenged, p. 108 
July, 1957 

- 41. Holloway, Margaret E. — We Used the Facts We 
Learned From 500 Interviews, p. 139 
42, Straus, Murray A.— What We Can Do for the P.irt^ 
- time Farmers, p. 153 
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OclobiT. 1957 • ■ 

43. Welch. Jr.. C. Herman— Daily Records: Chore or 
Kic-h Jlarvesf p. 203 
^ 44. iiarlow. Shirley—The Office •Sccrclarv— Envov of 
(iood Will. p. 20(i 
45. Welch. Jr.. C. Herman— Know Your Count v People, 
p. 2 lU • 

December, 1957 

4(). Sahrosky. Laurel K.— Evaluation as Related to r'uf)lic 
J{elation,s. p. 252 

January, 1958 

47. Leaj;ans. J. Taul-^Cive Your Kiir.ess a Cheekiip. p. (] 
March. H)5fl * 

, 4ij. Porter. Ward F. Studyini^ Our Human Resources. 

n. 57 • J 



April, 1058 

49. Hill. Kate Adelo— Thrce-Way Approach Pays Off. 
p. 79 

.May, '1958 

50. Carter. Glenn— Wants and Needs of Older Youth, 
p. 108 ' 

September, 1958 

51. Shipley, Fei n— lurching to Determine Xccds. .p. 18? 

N'oveniher, 1958 

52. Sarhau^h/ L. E.— Have You Tested Your Audience? 
p. 219 

January, 1959 

53. Frutchey. Fred P.- To Ciel . . . or Not to feet— 
Further Education, p. 3 . 
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